[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:13:03 dist-test-slave-7nm7 krb5kdc[17262](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:13:03 dist-test-slave-7nm7 krb5kdc[17262](info): set up 2 sockets
Aug 12 03:13:03 dist-test-slave-7nm7 krb5kdc[17262](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 03:13:04.428625 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.417s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:13:04 dist-test-slave-7nm7 krb5kdc[17262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:13:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:13:04.579119 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:42143
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:42143
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:04.714677 17287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:04.714953 17287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:04.716112 17287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:04.722730 17287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:04.722836 17287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:04.722859 17287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:04.722877 17287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:04.722956 17287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:13:04.729851 17287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:42143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:42143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:04.731086 17287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:04.732167 17287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:04.741556 17298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:04.741560 17297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:04.741683 17305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:04.741750 17287 server_base.cc:1034] running on GCE node
I20260812 03:13:04.742551 17287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:04.743633 17287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:04.744804 17287 hybrid_clock.cc:648] HybridClock initialized: now 1786504384744747 us; error 68 us; skew 500 ppm
Aug 12 03:13:04 dist-test-slave-7nm7 krb5kdc[17262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504384, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:04.748230 17287 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:13:04.749164 17287 webserver.cc:460] Webserver started at http://127.16.217.126:39685/ using document root <none> and password file <none>
I20260812 03:13:04.749748 17287 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:04.749812 17287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:04.750037 17287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:04.752346 17287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "a0eebef97ae24db49213c6a5320fa676"
format_stamp: "Formatted at 2026-08-12 03:13:04 on dist-test-slave-7nm7"
server_key: "de96b5c706f638c60dcac2aecc72fb89"
server_key_iv: "3fc15104ac998de1caa135ea42e8d987"
server_key_version: "encryptionkey@0"
I20260812 03:13:04.752964 17287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "a0eebef97ae24db49213c6a5320fa676"
format_stamp: "Formatted at 2026-08-12 03:13:04 on dist-test-slave-7nm7"
server_key: "de96b5c706f638c60dcac2aecc72fb89"
server_key_iv: "3fc15104ac998de1caa135ea42e8d987"
server_key_version: "encryptionkey@0"
I20260812 03:13:04.756827 17287 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:13:04.759368 17319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:04.760620 17287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:13:04.760759 17287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "a0eebef97ae24db49213c6a5320fa676"
format_stamp: "Formatted at 2026-08-12 03:13:04 on dist-test-slave-7nm7"
server_key: "de96b5c706f638c60dcac2aecc72fb89"
server_key_iv: "3fc15104ac998de1caa135ea42e8d987"
server_key_version: "encryptionkey@0"
I20260812 03:13:04.760867 17287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:04.792510 17287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:04.795826 17287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:04.796046 17287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:04.803833 17287 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:42143
I20260812 03:13:04.803865 17415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:42143 every 8 connection(s)
I20260812 03:13:04.804826 17287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:04.807638 17416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:04.808471 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 17287
I20260812 03:13:04.808661 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:13:04.810124 17253 external_mini_cluster.cc:1428] Setting key f4bc9fed2cdc12ec27e0e884e658d1a3
I20260812 03:13:04.814143 17416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676: Bootstrap starting.
I20260812 03:13:04.816730 17416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:04.817497 17416 log.cc:824] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:04.819867 17416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676: No bootstrap required, opened a new log
I20260812 03:13:04.823901 17416 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0eebef97ae24db49213c6a5320fa676" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 42143 } }
I20260812 03:13:04.824132 17416 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:04.824213 17416 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0eebef97ae24db49213c6a5320fa676, State: Initialized, Role: FOLLOWER
I20260812 03:13:04.824697 17416 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0eebef97ae24db49213c6a5320fa676" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 42143 } }
I20260812 03:13:04.824870 17416 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:04.824939 17416 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:04.825069 17416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:04.825956 17416 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0eebef97ae24db49213c6a5320fa676" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 42143 } }
I20260812 03:13:04.826407 17416 leader_election.cc:302] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0eebef97ae24db49213c6a5320fa676; no voters:
I20260812 03:13:04.826754 17416 leader_election.cc:288] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:13:04.826848 17426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:04.827037 17426 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676 [term 1 LEADER]: Becoming Leader. State: Replica: a0eebef97ae24db49213c6a5320fa676, State: Running, Role: LEADER
I20260812 03:13:04.827374 17426 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0eebef97ae24db49213c6a5320fa676" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 42143 } }
I20260812 03:13:04.828083 17416 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676 [sys.catalog]: configured and running, proceeding with master startup.
Aug 12 03:13:04 dist-test-slave-7nm7 krb5kdc[17262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:04.829154 17428 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0eebef97ae24db49213c6a5320fa676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0eebef97ae24db49213c6a5320fa676" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 42143 } } }
I20260812 03:13:04.829304 17428 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:04.829605 17429 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0eebef97ae24db49213c6a5320fa676. Latest consensus state: current_term: 1 leader_uuid: "a0eebef97ae24db49213c6a5320fa676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0eebef97ae24db49213c6a5320fa676" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 42143 } } }
I20260812 03:13:04.829700 17429 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676 [sys.catalog]: This master's current role is: LEADER
W20260812 03:13:04.833463 17457 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:13:04.833554 17457 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:13:04.833657 17447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:04.834523 17447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:04.836455 17420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:04.813380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34358 (local address 127.16.217.126:42143)
0812 03:13:04.813816 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:04.813833 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:04.814991 (+ 1158us) server_negotiation.cc:407] Connection header received
0812 03:13:04.818506 (+ 3515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:04.818527 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:04.818909 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:04.819237 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:04.821376 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:04.823006 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:04.824203 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:04.824729 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:04.824917 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:04.829933 (+ 5016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:04.829940 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:04.829962 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:04.830007 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:04.832291 (+ 2284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:04.833002 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:04.833009 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:04.833015 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:04.833167 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:04.833599 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:04.833605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:04.833609 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:04.834253 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:04.834465 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:04.835172 (+ 707us) server_negotiation.cc:299] Negotiation successful
0812 03:13:04.835477 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":145,"threads_started":1}
I20260812 03:13:04.840647 17447 catalog_manager.cc:1347] Generated new cluster ID: 6a8038aac50c4c99803ea6af73fcb0f0
I20260812 03:13:04.840723 17447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:04.862263 17447 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:13:04.863668 17447 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:04.876829 17447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0eebef97ae24db49213c6a5320fa676: Generated new TSK 0
I20260812 03:13:04.877713 17447 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
I20260812 03:13:04.917905 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:42143
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:05.045516 17468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:05.045763 17468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:05.045807 17468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:05.049860 17468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:05.049911 17468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:05.050014 17468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:13:05.055460 17468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:42143
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:05.056479 17468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:05.057504 17468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:05.067297 17482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:05.067351 17475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:05.067351 17480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:05.067481 17468 server_base.cc:1034] running on GCE node
I20260812 03:13:05.067932 17468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:05.068643 17468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:05.069845 17468 hybrid_clock.cc:648] HybridClock initialized: now 1786504385069829 us; error 29 us; skew 500 ppm
Aug 12 03:13:05 dist-test-slave-7nm7 krb5kdc[17262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504385, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:05.073184 17468 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:13:05.074205 17468 webserver.cc:460] Webserver started at http://127.16.217.65:35091/ using document root <none> and password file <none>
I20260812 03:13:05.074883 17468 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:05.074985 17468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:05.075246 17468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:05.077384 17468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "534d89d554fd4041a895122acd997caa"
format_stamp: "Formatted at 2026-08-12 03:13:05 on dist-test-slave-7nm7"
server_key: "1b6ba0fb7ae690258f61c0a13690e824"
server_key_iv: "731f143f8bbbb4448e4a32dc407adfd3"
server_key_version: "encryptionkey@0"
I20260812 03:13:05.078001 17468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "534d89d554fd4041a895122acd997caa"
format_stamp: "Formatted at 2026-08-12 03:13:05 on dist-test-slave-7nm7"
server_key: "1b6ba0fb7ae690258f61c0a13690e824"
server_key_iv: "731f143f8bbbb4448e4a32dc407adfd3"
server_key_version: "encryptionkey@0"
I20260812 03:13:05.082499 17468 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:13:05.085198 17495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:05.086687 17468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:05.086810 17468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "534d89d554fd4041a895122acd997caa"
format_stamp: "Formatted at 2026-08-12 03:13:05 on dist-test-slave-7nm7"
server_key: "1b6ba0fb7ae690258f61c0a13690e824"
server_key_iv: "731f143f8bbbb4448e4a32dc407adfd3"
server_key_version: "encryptionkey@0"
I20260812 03:13:05.086911 17468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:05.111881 17468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:05.115302 17468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:05.115566 17468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:05.116200 17468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:05.117242 17468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:05.117300 17468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:05.117347 17468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:05.117362 17468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:05.126632 17468 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:34355
I20260812 03:13:05.126653 17707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:34355 every 8 connection(s)
I20260812 03:13:05.127591 17468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:13:05.136695 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 17468
I20260812 03:13:05.136881 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:13:05.137185 17253 external_mini_cluster.cc:1428] Setting key 31418ad150ccba0fa54bea8b1cbac20e
Aug 12 03:13:05 dist-test-slave-7nm7 krb5kdc[17262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504385, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:05.143370 17420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:05.129794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:51031 (local address 127.16.217.126:42143)
0812 03:13:05.129938 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:05.129942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:05.131149 (+ 1207us) server_negotiation.cc:407] Connection header received
0812 03:13:05.132011 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:05.132014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:05.132074 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:05.132152 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:05.133342 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:05.133899 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:05.135113 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:05.135385 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:05.135482 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:05.138891 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:05.138896 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:05.138900 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:05.138930 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:05.140484 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:05.141128 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:05.141133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:05.141135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:05.141239 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:05.141844 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:05.141847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:05.141848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:05.142154 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:05.142361 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:05.143070 (+ 709us) server_negotiation.cc:299] Negotiation successful
0812 03:13:05.143214 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:13:05.144644 17711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:05.130328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:42143 (local address 127.16.217.65:51031)
0812 03:13:05.131011 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:05.131051 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:05.131819 (+ 768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:05.132324 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:05.132330 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:05.132687 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:05.133181 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:05.133195 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:05.134057 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:05.134059 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:05.134977 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:05.134988 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:05.135542 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:05.135545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:05.135703 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:05.136502 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:05.136519 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:05.138697 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:05.140618 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:05.140625 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:05.140644 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:05.140966 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:05.141342 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:05.141346 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:05.141350 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:05.141554 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:05.142362 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:05.142371 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:05.142743 (+ 372us) client_negotiation.cc:769] Sending connection context
0812 03:13:05.142971 (+ 228us) client_negotiation.cc:240] Negotiation successful
0812 03:13:05.143382 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":130,"threads_started":1}
I20260812 03:13:05.146163 17708 heartbeater.cc:347] Connected to a master server at 127.16.217.126:42143
I20260812 03:13:05.146471 17708 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:05.147020 17708 heartbeater.cc:511] Master 127.16.217.126:42143 requested a full tablet report, sending...
I20260812 03:13:05.148703 17349 ts_manager.cc:194] Registered new tserver with Master: 534d89d554fd4041a895122acd997caa (127.16.217.65:34355)
WARNING: no policy specified for kudu/127.16.217.66@KRBTEST.COM; defaulting to no policy
I20260812 03:13:05.149998 17349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:51031
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
I20260812 03:13:05.171471 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:42143
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:05.294378 17719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:05.294720 17719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:05.294831 17719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:05.299191 17719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:05.299294 17719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:05.299392 17719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:13:05.304895 17719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:42143
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:05.306255 17719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:05.307507 17719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:05.317731 17730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:05.317761 17731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:05.317731 17736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:05.318195 17719 server_base.cc:1034] running on GCE node
I20260812 03:13:05.318684 17719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:05.319432 17719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:05.320603 17719 hybrid_clock.cc:648] HybridClock initialized: now 1786504385320569 us; error 44 us; skew 500 ppm
Aug 12 03:13:05 dist-test-slave-7nm7 krb5kdc[17262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504385, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:05.323427 17719 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:13:05.324348 17719 webserver.cc:460] Webserver started at http://127.16.217.66:45525/ using document root <none> and password file <none>
I20260812 03:13:05.324976 17719 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:05.325042 17719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:05.325260 17719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:05.327422 17719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "975fd7c7898446f79b5423dffc571be6"
format_stamp: "Formatted at 2026-08-12 03:13:05 on dist-test-slave-7nm7"
server_key: "629d9e4fdcb7a59a74fd455f999ebf3a"
server_key_iv: "a1d1e6c7380972aae27b56f60e022ba8"
server_key_version: "encryptionkey@0"
I20260812 03:13:05.328047 17719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "975fd7c7898446f79b5423dffc571be6"
format_stamp: "Formatted at 2026-08-12 03:13:05 on dist-test-slave-7nm7"
server_key: "629d9e4fdcb7a59a74fd455f999ebf3a"
server_key_iv: "a1d1e6c7380972aae27b56f60e022ba8"
server_key_version: "encryptionkey@0"
I20260812 03:13:05.332163 17719 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:13:05.334704 17754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:05.336164 17719 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:05.336323 17719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "975fd7c7898446f79b5423dffc571be6"
format_stamp: "Formatted at 2026-08-12 03:13:05 on dist-test-slave-7nm7"
server_key: "629d9e4fdcb7a59a74fd455f999ebf3a"
server_key_iv: "a1d1e6c7380972aae27b56f60e022ba8"
server_key_version: "encryptionkey@0"
I20260812 03:13:05.336448 17719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:05.352490 17719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:05.356550 17719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:05.356807 17719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:05.357450 17719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:05.358399 17719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:05.358465 17719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:05.358537 17719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:05.358605 17719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:05.369180 17719 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:43557
I20260812 03:13:05.369221 17972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:43557 every 8 connection(s)
I20260812 03:13:05.370107 17719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:13:05.380239 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 17719
I20260812 03:13:05.380348 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:13:05.380666 17253 external_mini_cluster.cc:1428] Setting key 48b7b465f69d8fb05ed76f75b3b49510
Aug 12 03:13:05 dist-test-slave-7nm7 krb5kdc[17262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504385, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:05.386050 17420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:05.372062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:50047 (local address 127.16.217.126:42143)
0812 03:13:05.372384 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:05.372390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:05.373390 (+ 1000us) server_negotiation.cc:407] Connection header received
0812 03:13:05.374433 (+ 1043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:05.374438 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:05.374518 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:05.374640 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:05.376023 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:05.376793 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:05.378290 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:05.378549 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:05.378662 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:05.381517 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:05.381520 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:05.381523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:05.381582 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:05.383357 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:05.384033 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:05.384036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:05.384038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:05.384145 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:05.384586 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:05.384590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:05.384592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:05.384918 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:05.385041 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:05.385660 (+ 619us) server_negotiation.cc:299] Negotiation successful
0812 03:13:05.385844 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260812 03:13:05.386812 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:05.372458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:42143 (local address 127.16.217.66:50047)
0812 03:13:05.373222 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:05.373267 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:05.374083 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:05.374841 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:05.374847 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:05.375264 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:05.375780 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:05.375795 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:05.377018 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:05.377026 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:05.378096 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:05.378107 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:05.378725 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:05.378729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:05.378874 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:05.379513 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:05.379532 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:05.381329 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:05.383485 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:05.383494 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:05.383516 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:05.383919 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:05.384243 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:05.384248 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:05.384253 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:05.384453 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:05.385060 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:05.385066 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:05.385357 (+ 291us) client_negotiation.cc:769] Sending connection context
0812 03:13:05.385539 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 03:13:05.385809 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":120,"threads_started":1}
I20260812 03:13:05.388026 17973 heartbeater.cc:347] Connected to a master server at 127.16.217.126:42143
I20260812 03:13:05.388315 17973 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:05.388980 17973 heartbeater.cc:511] Master 127.16.217.126:42143 requested a full tablet report, sending...
I20260812 03:13:05.390347 17349 ts_manager.cc:194] Registered new tserver with Master: 975fd7c7898446f79b5423dffc571be6 (127.16.217.66:43557)
I20260812 03:13:05.391004 17349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:50047
WARNING: no policy specified for kudu/127.16.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
I20260812 03:13:05.413164 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:42143
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:05.542482 17987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:05.542760 17987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:05.542807 17987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:05.546852 17987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:05.546911 17987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:05.546973 17987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:13:05.552628 17987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:42143
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:05.553723 17987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:05.554760 17987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:05.564944 18004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:05.564957 18000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:05.565083 17998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:05.565332 17987 server_base.cc:1034] running on GCE node
I20260812 03:13:05.565791 17987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:05.566493 17987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:05.567727 17987 hybrid_clock.cc:648] HybridClock initialized: now 1786504385567704 us; error 36 us; skew 500 ppm
Aug 12 03:13:05 dist-test-slave-7nm7 krb5kdc[17262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504385, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:05.571259 17987 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:13:05.572247 17987 webserver.cc:460] Webserver started at http://127.16.217.67:39287/ using document root <none> and password file <none>
I20260812 03:13:05.572865 17987 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:05.572962 17987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:05.573220 17987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:05.575508 17987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "888afcaf55414a949a30958252396741"
format_stamp: "Formatted at 2026-08-12 03:13:05 on dist-test-slave-7nm7"
server_key: "6ec6ce13fcf5815c98d6fdcdfbd1ea9a"
server_key_iv: "6591029cad7491067dbbeeaa548d638e"
server_key_version: "encryptionkey@0"
I20260812 03:13:05.576112 17987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "888afcaf55414a949a30958252396741"
format_stamp: "Formatted at 2026-08-12 03:13:05 on dist-test-slave-7nm7"
server_key: "6ec6ce13fcf5815c98d6fdcdfbd1ea9a"
server_key_iv: "6591029cad7491067dbbeeaa548d638e"
server_key_version: "encryptionkey@0"
I20260812 03:13:05.581113 17987 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 03:13:05.584479 18014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:05.586002 17987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:05.586289 17987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "888afcaf55414a949a30958252396741"
format_stamp: "Formatted at 2026-08-12 03:13:05 on dist-test-slave-7nm7"
server_key: "6ec6ce13fcf5815c98d6fdcdfbd1ea9a"
server_key_iv: "6591029cad7491067dbbeeaa548d638e"
server_key_version: "encryptionkey@0"
I20260812 03:13:05.586508 17987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:05.599768 17987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:05.603252 17987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:05.603412 17987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:05.603956 17987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:05.604772 17987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:05.604820 17987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:05.604861 17987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:05.604875 17987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:05.614779 17987 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:38201
I20260812 03:13:05.614800 18239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:38201 every 8 connection(s)
I20260812 03:13:05.615666 17987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:13:05.621475 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 17987
I20260812 03:13:05.621608 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:13:05.621886 17253 external_mini_cluster.cc:1428] Setting key 44ece439d6dfab76b2fcd7e7d1fbc0b0
Aug 12 03:13:05 dist-test-slave-7nm7 krb5kdc[17262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504385, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:05.629873 17420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:05.617940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:53181 (local address 127.16.217.126:42143)
0812 03:13:05.618100 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:05.618104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:05.619255 (+ 1151us) server_negotiation.cc:407] Connection header received
0812 03:13:05.620076 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:05.620079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:05.620141 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:05.620218 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:05.621458 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:05.622286 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:05.623813 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:05.624021 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:05.624101 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:05.626708 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:05.626711 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:05.626713 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:05.626737 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:05.627921 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:05.628392 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:05.628395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:05.628396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:05.628469 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:05.628863 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:05.628865 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:05.628866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:05.629036 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:05.629111 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:05.629654 (+ 543us) server_negotiation.cc:299] Negotiation successful
0812 03:13:05.629752 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 03:13:05.630722 18244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:05.618420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:42143 (local address 127.16.217.67:53181)
0812 03:13:05.619105 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:05.619144 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:05.619887 (+ 743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:05.620376 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:05.620383 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:05.620739 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:05.621306 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:05.621320 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:05.622432 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:05.622436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:05.623707 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:05.623718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:05.624172 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:05.624176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:05.624384 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:05.625071 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:05.625088 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:05.626560 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:05.628026 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:05.628037 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:05.628049 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:05.628304 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:05.628596 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:05.628600 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:05.628604 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:05.628776 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:05.629122 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:05.629127 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:05.629365 (+ 238us) client_negotiation.cc:769] Sending connection context
0812 03:13:05.629589 (+ 224us) client_negotiation.cc:240] Negotiation successful
0812 03:13:05.629830 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":132,"threads_started":1}
I20260812 03:13:05.631851 18241 heartbeater.cc:347] Connected to a master server at 127.16.217.126:42143
I20260812 03:13:05.632050 18241 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:05.632475 18241 heartbeater.cc:511] Master 127.16.217.126:42143 requested a full tablet report, sending...
I20260812 03:13:05.633517 17349 ts_manager.cc:194] Registered new tserver with Master: 888afcaf55414a949a30958252396741 (127.16.217.67:38201)
I20260812 03:13:05.634049 17349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:53181
I20260812 03:13:05.636474 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:13:05 dist-test-slave-7nm7 krb5kdc[17262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504385, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:13:05 dist-test-slave-7nm7 krb5kdc[17262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504385, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.217.65@KRBTEST.COM
I20260812 03:13:05.666599 18256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:05.656441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39478 (local address 127.16.217.65:34355)
0812 03:13:05.656805 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:05.656814 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:05.656844 (+ 30us) server_negotiation.cc:407] Connection header received
0812 03:13:05.656915 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:05.656921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:05.657125 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:05.657291 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:05.658018 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:05.658871 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:05.659738 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:05.660115 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:05.660241 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:05.662864 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:05.662874 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:05.662890 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:05.662916 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:05.664612 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:05.665083 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:05.665090 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:05.665092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:05.665190 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:05.665477 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:05.665482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:05.665484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:05.665804 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:05.666030 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:05.666213 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 03:13:05.666398 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":118,"threads_started":1}
W20260812 03:13:05.667641 17543 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39478
Aug 12 03:13:05 dist-test-slave-7nm7 krb5kdc[17262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:13:05 dist-test-slave-7nm7 krb5kdc[17262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.65@KRBTEST.COM
I20260812 03:13:05.698107 18256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:05.689158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39494 (local address 127.16.217.65:34355)
0812 03:13:05.689329 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:05.689336 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:05.689458 (+ 122us) server_negotiation.cc:407] Connection header received
0812 03:13:05.689666 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:05.689670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:05.689722 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:05.689822 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:05.690750 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:05.691275 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:05.692161 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:05.692392 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:05.692514 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:05.695206 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:05.695210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:05.695212 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:05.695235 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:05.696452 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:05.696874 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:05.696877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:05.696879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:05.696958 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:05.697244 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:05.697250 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:05.697252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:05.697417 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:05.697554 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:05.697800 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 03:13:05.697930 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:13:05.699384 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 17468
I20260812 03:13:05.714397 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 17719
I20260812 03:13:05.734696 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 17987
I20260812 03:13:05.748448 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 17287
2026-08-12T03:13:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2779 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:13:05 dist-test-slave-7nm7 krb5kdc[18273](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:13:05 dist-test-slave-7nm7 krb5kdc[18273](info): set up 2 sockets
Aug 12 03:13:05 dist-test-slave-7nm7 krb5kdc[18273](info): commencing operation
krb5kdc: starting...
W20260812 03:13:07.085371 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:13:07 dist-test-slave-7nm7 krb5kdc[18273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:13:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:13:07.216791 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:44505
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:42881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:44505
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:07.337566 18301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:07.337844 18301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:07.337893 18301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:07.341982 18301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:07.342042 18301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:07.342061 18301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:07.342079 18301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:07.342094 18301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:13:07.347590 18301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:42881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:44505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:44505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:07.348662 18301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:07.349689 18301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:07.359021 18309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:07.359038 18310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:07.359045 18315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:07.359092 18301 server_base.cc:1034] running on GCE node
I20260812 03:13:07.359855 18301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:07.360896 18301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:07.362067 18301 hybrid_clock.cc:648] HybridClock initialized: now 1786504387362041 us; error 41 us; skew 500 ppm
Aug 12 03:13:07 dist-test-slave-7nm7 krb5kdc[18273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504387, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:07.365087 18301 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:13:07.365934 18301 webserver.cc:460] Webserver started at http://127.16.217.126:44215/ using document root <none> and password file <none>
I20260812 03:13:07.366544 18301 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:07.366607 18301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:07.366780 18301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:07.368764 18301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "1ea866a814154a59892c4c077bed46ef"
format_stamp: "Formatted at 2026-08-12 03:13:07 on dist-test-slave-7nm7"
server_key: "ed6cae0e517e32ebab363fda3aa7fe2c"
server_key_iv: "354d4fdddabe4d65a318fff796e5497d"
server_key_version: "encryptionkey@0"
I20260812 03:13:07.369283 18301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "1ea866a814154a59892c4c077bed46ef"
format_stamp: "Formatted at 2026-08-12 03:13:07 on dist-test-slave-7nm7"
server_key: "ed6cae0e517e32ebab363fda3aa7fe2c"
server_key_iv: "354d4fdddabe4d65a318fff796e5497d"
server_key_version: "encryptionkey@0"
I20260812 03:13:07.372972 18301 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 03:13:07.375458 18328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:07.376659 18301 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:13:07.376818 18301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "1ea866a814154a59892c4c077bed46ef"
format_stamp: "Formatted at 2026-08-12 03:13:07 on dist-test-slave-7nm7"
server_key: "ed6cae0e517e32ebab363fda3aa7fe2c"
server_key_iv: "354d4fdddabe4d65a318fff796e5497d"
server_key_version: "encryptionkey@0"
I20260812 03:13:07.376926 18301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:07.402441 18301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:07.405925 18301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:07.406268 18301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:07.413573 18301 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:44505
I20260812 03:13:07.413611 18423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:44505 every 8 connection(s)
I20260812 03:13:07.414567 18301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:07.417440 18427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:07.424167 18427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef: Bootstrap starting.
I20260812 03:13:07.424682 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 18301
I20260812 03:13:07.424826 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:13:07.425141 17253 external_mini_cluster.cc:1428] Setting key c74684247b5418c1811c15f0108dd406
I20260812 03:13:07.427408 18427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:07.428220 18427 log.cc:824] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:07.430811 18427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef: No bootstrap required, opened a new log
Aug 12 03:13:07 dist-test-slave-7nm7 krb5kdc[18273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:07.435851 18427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea866a814154a59892c4c077bed46ef" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44505 } }
I20260812 03:13:07.436138 18427 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:07.436224 18427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ea866a814154a59892c4c077bed46ef, State: Initialized, Role: FOLLOWER
I20260812 03:13:07.436728 18427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea866a814154a59892c4c077bed46ef" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44505 } }
I20260812 03:13:07.436981 18427 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:07.437074 18427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:07.437216 18427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:07.438746 18427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea866a814154a59892c4c077bed46ef" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44505 } }
I20260812 03:13:07.439181 18427 leader_election.cc:302] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ea866a814154a59892c4c077bed46ef; no voters:
I20260812 03:13:07.439561 18427 leader_election.cc:288] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:13:07.439703 18439 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:07.439970 18439 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef [term 1 LEADER]: Becoming Leader. State: Replica: 1ea866a814154a59892c4c077bed46ef, State: Running, Role: LEADER
I20260812 03:13:07.440375 18439 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea866a814154a59892c4c077bed46ef" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44505 } }
I20260812 03:13:07.441061 18427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:07.442474 18441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ea866a814154a59892c4c077bed46ef. Latest consensus state: current_term: 1 leader_uuid: "1ea866a814154a59892c4c077bed46ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea866a814154a59892c4c077bed46ef" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44505 } } }
I20260812 03:13:07.442683 18441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:07.442832 18437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:07.426838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51042 (local address 127.16.217.126:44505)
0812 03:13:07.427336 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:07.427352 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:07.427389 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:13:07.428138 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:07.428156 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:07.428645 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:07.429150 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:07.430065 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:07.431370 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:07.432621 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:07.433115 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:07.433264 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:07.436530 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:07.436539 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:07.436569 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:07.436613 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:07.439072 (+ 2459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:07.439656 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:07.439663 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:07.439670 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:07.439827 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:07.440204 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:07.440207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:07.440208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:07.440738 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:07.440942 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:07.441264 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 03:13:07.441592 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":156,"threads_started":1}
I20260812 03:13:07.442123 18440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ea866a814154a59892c4c077bed46ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea866a814154a59892c4c077bed46ef" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44505 } } }
I20260812 03:13:07.442970 18440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef [sys.catalog]: This master's current role is: LEADER
W20260812 03:13:07.446547 18465 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:13:07.446625 18465 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:13:07.446712 18457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:07.447425 18457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:07.453374 18457 catalog_manager.cc:1347] Generated new cluster ID: d82bf47549b14b3d8ba239283080fb5b
I20260812 03:13:07.453446 18457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:07.460388 18457 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:13:07.461401 18457 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:07.469256 18457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ea866a814154a59892c4c077bed46ef: Generated new TSK 0
I20260812 03:13:07.470001 18457 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
I20260812 03:13:07.507406 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:44505
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:42881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:07.624202 18472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:07.624480 18472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:07.624557 18472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:07.628579 18472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:07.628672 18472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:07.628764 18472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:13:07.634100 18472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:42881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:44505
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:07.635264 18472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:07.636337 18472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:07.646961 18480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:07.646965 18482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:07.647111 18472 server_base.cc:1034] running on GCE node
W20260812 03:13:07.647300 18487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:07.647696 18472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:07.648379 18472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:07.649565 18472 hybrid_clock.cc:648] HybridClock initialized: now 1786504387649532 us; error 44 us; skew 500 ppm
Aug 12 03:13:07 dist-test-slave-7nm7 krb5kdc[18273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504387, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:07.653270 18472 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:13:07.654321 18472 webserver.cc:460] Webserver started at http://127.16.217.65:33703/ using document root <none> and password file <none>
I20260812 03:13:07.654953 18472 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:07.655045 18472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:07.655297 18472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:07.657440 18472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "948c005970b84ae88bb3b0df9b220279"
format_stamp: "Formatted at 2026-08-12 03:13:07 on dist-test-slave-7nm7"
server_key: "0489ded6aeb2169d9dd02736cd4263df"
server_key_iv: "86448b96827193b28b2f0321d6cdc03b"
server_key_version: "encryptionkey@0"
I20260812 03:13:07.658134 18472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "948c005970b84ae88bb3b0df9b220279"
format_stamp: "Formatted at 2026-08-12 03:13:07 on dist-test-slave-7nm7"
server_key: "0489ded6aeb2169d9dd02736cd4263df"
server_key_iv: "86448b96827193b28b2f0321d6cdc03b"
server_key_version: "encryptionkey@0"
I20260812 03:13:07.662179 18472 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260812 03:13:07.664932 18502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:07.666355 18472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:07.666482 18472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "948c005970b84ae88bb3b0df9b220279"
format_stamp: "Formatted at 2026-08-12 03:13:07 on dist-test-slave-7nm7"
server_key: "0489ded6aeb2169d9dd02736cd4263df"
server_key_iv: "86448b96827193b28b2f0321d6cdc03b"
server_key_version: "encryptionkey@0"
I20260812 03:13:07.666597 18472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:07.691234 18472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:07.695622 18472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:07.695851 18472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:07.696486 18472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:07.697396 18472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:07.697445 18472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:07.697510 18472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:07.697541 18472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:07.706737 18472 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:36715
I20260812 03:13:07.706774 18717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:36715 every 8 connection(s)
I20260812 03:13:07.707672 18472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:13:07.715500 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 18472
I20260812 03:13:07.715652 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:13:07.715929 17253 external_mini_cluster.cc:1428] Setting key 2ea3f4fc84983cb7b7fa0d1ce76849f5
Aug 12 03:13:07 dist-test-slave-7nm7 krb5kdc[18273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504387, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:07.723893 18437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:07.709967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:53339 (local address 127.16.217.126:44505)
0812 03:13:07.710156 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:07.710162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:07.711242 (+ 1080us) server_negotiation.cc:407] Connection header received
0812 03:13:07.712719 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:07.712722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:07.712773 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:07.712854 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:07.714064 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:07.714698 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:07.716585 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:07.716960 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:07.717059 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:07.720058 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:07.720061 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:07.720063 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:07.720105 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:07.721726 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:07.722299 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:07.722305 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:07.722307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:07.722388 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:07.722791 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:07.722795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:07.722796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:07.722977 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:07.723113 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:07.723605 (+ 492us) server_negotiation.cc:299] Negotiation successful
0812 03:13:07.723728 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 03:13:07.724733 18722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:07.710547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:44505 (local address 127.16.217.65:53339)
0812 03:13:07.711065 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:07.711111 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:07.712490 (+ 1379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:07.713015 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:07.713022 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:07.713416 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:07.713904 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:07.713917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:07.714839 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:07.714843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:07.716438 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:07.716449 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:07.717137 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:07.717141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:07.717339 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:07.718070 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:07.718090 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:07.719880 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:07.721850 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:07.721856 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:07.721872 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:07.722153 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:07.722486 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:07.722490 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:07.722492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:07.722699 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:07.723091 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:07.723098 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:07.723366 (+ 268us) client_negotiation.cc:769] Sending connection context
0812 03:13:07.723562 (+ 196us) client_negotiation.cc:240] Negotiation successful
0812 03:13:07.723901 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":124,"threads_started":1}
I20260812 03:13:07.725986 18720 heartbeater.cc:347] Connected to a master server at 127.16.217.126:44505
I20260812 03:13:07.726308 18720 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.16.217.66@KRBTEST.COM; defaulting to no policy
I20260812 03:13:07.726933 18720 heartbeater.cc:511] Master 127.16.217.126:44505 requested a full tablet report, sending...
I20260812 03:13:07.728648 18350 ts_manager.cc:194] Registered new tserver with Master: 948c005970b84ae88bb3b0df9b220279 (127.16.217.65:36715)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.66@KRBTEST.COM" created.
I20260812 03:13:07.729990 18350 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:53339
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
I20260812 03:13:07.745896 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:44505
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:42881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:07.856256 18726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:07.856559 18726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:07.856637 18726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:07.860677 18726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:07.860772 18726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:07.860860 18726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:13:07.866263 18726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:42881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:44505
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:07.867347 18726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:07.868376 18726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:07.879277 18733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:07.879357 18734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:07.879357 18736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:07.879767 18726 server_base.cc:1034] running on GCE node
I20260812 03:13:07.880214 18726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:07.880865 18726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:07.882035 18726 hybrid_clock.cc:648] HybridClock initialized: now 1786504387882033 us; error 54 us; skew 500 ppm
Aug 12 03:13:07 dist-test-slave-7nm7 krb5kdc[18273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504387, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:07.884979 18726 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:13:07.886108 18726 webserver.cc:460] Webserver started at http://127.16.217.66:43345/ using document root <none> and password file <none>
I20260812 03:13:07.886718 18726 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:07.886781 18726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:07.886947 18726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:07.888926 18726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "e65a5a8a61e4404e8f920b9cf27e1401"
format_stamp: "Formatted at 2026-08-12 03:13:07 on dist-test-slave-7nm7"
server_key: "183da555ec3ce9f76e3bf12562843efd"
server_key_iv: "2be32b10e7f9e5d063ce53e1dc07fe75"
server_key_version: "encryptionkey@0"
I20260812 03:13:07.889441 18726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "e65a5a8a61e4404e8f920b9cf27e1401"
format_stamp: "Formatted at 2026-08-12 03:13:07 on dist-test-slave-7nm7"
server_key: "183da555ec3ce9f76e3bf12562843efd"
server_key_iv: "2be32b10e7f9e5d063ce53e1dc07fe75"
server_key_version: "encryptionkey@0"
I20260812 03:13:07.893293 18726 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 03:13:07.895849 18757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:07.897138 18726 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:13:07.897260 18726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "e65a5a8a61e4404e8f920b9cf27e1401"
format_stamp: "Formatted at 2026-08-12 03:13:07 on dist-test-slave-7nm7"
server_key: "183da555ec3ce9f76e3bf12562843efd"
server_key_iv: "2be32b10e7f9e5d063ce53e1dc07fe75"
server_key_version: "encryptionkey@0"
I20260812 03:13:07.897354 18726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:07.911480 18726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:07.914499 18726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:07.914650 18726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:07.915179 18726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:07.915992 18726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:07.916061 18726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:07.916133 18726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:07.916159 18726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:07.924938 18726 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:36537
I20260812 03:13:07.924966 18950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:36537 every 8 connection(s)
I20260812 03:13:07.925876 18726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:13:07.932446 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 18726
I20260812 03:13:07.932585 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:13:07.932868 17253 external_mini_cluster.cc:1428] Setting key 32178f7fc616c3dd4411db0f48ae14d7
Aug 12 03:13:07 dist-test-slave-7nm7 krb5kdc[18273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504387, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:07.940377 18437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:07.927736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:55765 (local address 127.16.217.126:44505)
0812 03:13:07.927892 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:07.927895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:07.928714 (+ 819us) server_negotiation.cc:407] Connection header received
0812 03:13:07.929938 (+ 1224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:07.929941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:07.929988 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:07.930066 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:07.931226 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:07.931744 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:07.933046 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:07.933304 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:07.933419 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:07.936203 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:07.936207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:07.936209 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:07.936256 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:07.938024 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:07.938658 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:07.938661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:07.938662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:07.938739 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:07.939177 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:07.939185 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:07.939186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:07.939426 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:07.939507 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:07.940143 (+ 636us) server_negotiation.cc:299] Negotiation successful
0812 03:13:07.940255 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 03:13:07.941664 18955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:07.928089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:44505 (local address 127.16.217.66:55765)
0812 03:13:07.928547 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:07.928587 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:07.929746 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:07.930273 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:07.930285 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:07.930633 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:07.931098 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:07.931112 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:07.931878 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:07.931881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:07.932877 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:07.932887 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:07.933514 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:07.933519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:07.933713 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:07.934368 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:07.934389 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:07.936015 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:07.938135 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:07.938144 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:07.938162 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:07.938560 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:07.938833 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:07.938840 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:07.938844 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:07.939070 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:07.939543 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:07.939553 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:07.939868 (+ 315us) client_negotiation.cc:769] Sending connection context
0812 03:13:07.940096 (+ 228us) client_negotiation.cc:240] Negotiation successful
0812 03:13:07.940471 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":119,"threads_started":1}
I20260812 03:13:07.943084 18951 heartbeater.cc:347] Connected to a master server at 127.16.217.126:44505
I20260812 03:13:07.943408 18951 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.16.217.67@KRBTEST.COM; defaulting to no policy
I20260812 03:13:07.943950 18951 heartbeater.cc:511] Master 127.16.217.126:44505 requested a full tablet report, sending...
I20260812 03:13:07.945006 18350 ts_manager.cc:194] Registered new tserver with Master: e65a5a8a61e4404e8f920b9cf27e1401 (127.16.217.66:36537)
I20260812 03:13:07.945626 18350 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:55765
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
I20260812 03:13:07.961215 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:44505
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:42881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:08.078715 18961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:08.079027 18961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:08.079113 18961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:08.083436 18961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:08.083524 18961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:08.083618 18961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:13:08.089174 18961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:42881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:44505
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:08.090353 18961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:08.091810 18961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:08.101897 18969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:08.101945 18971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:08.101892 18974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:08.101997 18961 server_base.cc:1034] running on GCE node
I20260812 03:13:08.102639 18961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:08.103313 18961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:08.104491 18961 hybrid_clock.cc:648] HybridClock initialized: now 1786504388104472 us; error 32 us; skew 500 ppm
Aug 12 03:13:08 dist-test-slave-7nm7 krb5kdc[18273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504388, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:08.111588 18961 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:13:08.113123 18961 webserver.cc:460] Webserver started at http://127.16.217.67:33379/ using document root <none> and password file <none>
I20260812 03:13:08.113714 18961 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:08.113777 18961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:08.113952 18961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:08.116165 18961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "19a1655d556a44179d5e0ea877a6a920"
format_stamp: "Formatted at 2026-08-12 03:13:08 on dist-test-slave-7nm7"
server_key: "bfcf7968355b99822847810ccded63bf"
server_key_iv: "2ff8e9aecef40fce76850ca9220bc120"
server_key_version: "encryptionkey@0"
I20260812 03:13:08.116704 18961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "19a1655d556a44179d5e0ea877a6a920"
format_stamp: "Formatted at 2026-08-12 03:13:08 on dist-test-slave-7nm7"
server_key: "bfcf7968355b99822847810ccded63bf"
server_key_iv: "2ff8e9aecef40fce76850ca9220bc120"
server_key_version: "encryptionkey@0"
I20260812 03:13:08.120492 18961 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:13:08.122910 18983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:08.124181 18961 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:13:08.124344 18961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "19a1655d556a44179d5e0ea877a6a920"
format_stamp: "Formatted at 2026-08-12 03:13:08 on dist-test-slave-7nm7"
server_key: "bfcf7968355b99822847810ccded63bf"
server_key_iv: "2ff8e9aecef40fce76850ca9220bc120"
server_key_version: "encryptionkey@0"
I20260812 03:13:08.124451 18961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:08.140408 18961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:08.143774 18961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:08.144003 18961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:08.144631 18961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:08.145545 18961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:08.145598 18961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:08.145674 18961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:08.145712 18961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:08.155159 18961 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:45307
I20260812 03:13:08.155189 19209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:45307 every 8 connection(s)
I20260812 03:13:08.156136 18961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:13:08.158032 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 18961
I20260812 03:13:08.158160 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:13:08.158444 17253 external_mini_cluster.cc:1428] Setting key 95e553421f71b3a8026dab26e7c74995
Aug 12 03:13:08 dist-test-slave-7nm7 krb5kdc[18273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504388, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:08.169389 18437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:08.157977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:54391 (local address 127.16.217.126:44505)
0812 03:13:08.158141 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:08.158146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:08.159128 (+ 982us) server_negotiation.cc:407] Connection header received
0812 03:13:08.160750 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:08.160752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:08.160797 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:08.160863 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:08.161906 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.162439 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.163498 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.163708 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.163783 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:08.166086 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:08.166089 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:08.166091 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:08.166114 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:08.167718 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:08.168151 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:08.168154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:08.168155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:08.168227 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:08.168506 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:08.168508 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:08.168509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:08.168681 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:08.168756 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:08.169179 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 03:13:08.169268 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 03:13:08.170218 19212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:08.158417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:44505 (local address 127.16.217.67:54391)
0812 03:13:08.158975 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:08.159027 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:08.160511 (+ 1484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:08.160984 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:08.160991 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:08.161330 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:08.161780 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:08.161794 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.162546 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.162550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:08.163392 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:08.163402 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.163832 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.163834 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:08.163971 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:08.164500 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:08.164516 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:08.165933 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:08.167814 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:08.167818 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:08.167830 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:08.168066 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:08.168306 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:08.168308 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:08.168310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:08.168428 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:08.168766 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:08.168771 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:08.168981 (+ 210us) client_negotiation.cc:769] Sending connection context
0812 03:13:08.169128 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 03:13:08.169356 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":136,"threads_started":1}
I20260812 03:13:08.171648 19210 heartbeater.cc:347] Connected to a master server at 127.16.217.126:44505
I20260812 03:13:08.171934 19210 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:08.172587 19210 heartbeater.cc:511] Master 127.16.217.126:44505 requested a full tablet report, sending...
I20260812 03:13:08.173682 18350 ts_manager.cc:194] Registered new tserver with Master: 19a1655d556a44179d5e0ea877a6a920 (127.16.217.67:45307)
I20260812 03:13:08.174346 18350 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:54391
I20260812 03:13:08.184329 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:13:08.195541 18437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:08.187989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51050 (local address 127.16.217.126:44505)
0812 03:13:08.188165 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:08.188171 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:08.188392 (+ 221us) server_negotiation.cc:407] Connection header received
0812 03:13:08.188488 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:08.188490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:08.188537 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:08.188624 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:08.189314 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.189852 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.190735 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.190962 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.191062 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:08.192364 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:08.192368 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:08.192370 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:08.192393 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:08.193884 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:08.194360 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:08.194363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:08.194364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:08.194438 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:08.194701 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:08.194704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:08.194705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:08.194910 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:08.195014 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:08.195223 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 03:13:08.195312 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 03:13:08.200060 18350 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:13:08.203032 18350 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:13:08.218055 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:08.211877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57626 (local address 127.16.217.66:36537)
0812 03:13:08.213403 (+ 1526us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:08.213415 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:08.213438 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:13:08.213519 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:08.213527 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:08.213734 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:08.213918 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:08.214142 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.214962 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.216419 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.217375 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.217493 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:08.217650 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:08.217794 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 03:13:08.217887 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1424,"thread_start_us":195,"threads_started":1}
I20260812 03:13:08.218039 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:08.211685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:36537 (local address 127.0.0.1:57626)
0812 03:13:08.212248 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:08.212298 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:08.212450 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:08.213894 (+ 1444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:08.213897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:08.213915 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:08.214017 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:08.214024 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.215107 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.215113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:08.216273 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:08.216286 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.217496 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.217499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:08.217604 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:08.217653 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 03:13:08.217764 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 03:13:08.217888 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":1,"thread_start_us":73,"threads_started":1}
I20260812 03:13:08.218508 19229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:08.212425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:45307 (local address 127.0.0.1:33746)
0812 03:13:08.212902 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:08.212917 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:08.213009 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:08.213698 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:08.213701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:08.213717 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:08.213815 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:08.213820 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.215746 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.215749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:08.216965 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:08.216975 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.218084 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.218088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:08.218192 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:08.218218 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 03:13:08.218291 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:13:08.218362 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":59,"thread_start_us":119,"threads_started":1}
I20260812 03:13:08.218544 19242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:08.211891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:36715 (local address 127.0.0.1:35326)
0812 03:13:08.212693 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:08.212708 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:08.212802 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:08.213495 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:08.213503 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:08.213528 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:08.213699 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:08.213706 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.215747 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.215751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:08.217122 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:08.217144 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.218194 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.218198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:08.218321 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:08.218340 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 03:13:08.218382 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 03:13:08.218434 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"mutex_wait_us":116,"thread_start_us":117,"threads_started":1}
I20260812 03:13:08.219229 19243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:08.212473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35326 (local address 127.16.217.65:36715)
0812 03:13:08.212963 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:08.212984 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:08.213003 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:13:08.213099 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:08.213105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:08.213316 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:08.213852 (+ 536us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:08.214015 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.215573 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.217270 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.218062 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.218796 (+ 734us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:08.218936 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:08.218988 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 03:13:08.219075 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":128,"threads_started":1}
I20260812 03:13:08.219229 19247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:08.212876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33746 (local address 127.16.217.67:45307)
0812 03:13:08.213244 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:08.213254 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:08.213275 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:13:08.213346 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:08.213350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:08.213528 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:08.213948 (+ 420us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:08.214102 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.215573 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.217097 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.217929 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.218796 (+ 867us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:08.218936 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:08.218988 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 03:13:08.219103 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":158,"threads_started":1}
I20260812 03:13:08.221513 18830 tablet_service.cc:1467] Processing CreateTablet for tablet 6a5c906b7f90447d8295f417843127dd (DEFAULT_TABLE table=test-table [id=1cb0323fc2714294b9a8c807e0f755cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:08.221570 18591 tablet_service.cc:1467] Processing CreateTablet for tablet 6a5c906b7f90447d8295f417843127dd (DEFAULT_TABLE table=test-table [id=1cb0323fc2714294b9a8c807e0f755cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:08.222518 18591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a5c906b7f90447d8295f417843127dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:08.222527 18830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a5c906b7f90447d8295f417843127dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:08.225431 19073 tablet_service.cc:1467] Processing CreateTablet for tablet 6a5c906b7f90447d8295f417843127dd (DEFAULT_TABLE table=test-table [id=1cb0323fc2714294b9a8c807e0f755cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:08.226570 19073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a5c906b7f90447d8295f417843127dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:08.229251 19249 tablet_bootstrap.cc:492] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401: Bootstrap starting.
I20260812 03:13:08.230566 19250 tablet_bootstrap.cc:492] T 6a5c906b7f90447d8295f417843127dd P 948c005970b84ae88bb3b0df9b220279: Bootstrap starting.
I20260812 03:13:08.231596 19249 tablet_bootstrap.cc:654] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:08.232738 19250 tablet_bootstrap.cc:654] T 6a5c906b7f90447d8295f417843127dd P 948c005970b84ae88bb3b0df9b220279: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:08.232908 19249 log.cc:824] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:08.232926 19251 tablet_bootstrap.cc:492] T 6a5c906b7f90447d8295f417843127dd P 19a1655d556a44179d5e0ea877a6a920: Bootstrap starting.
I20260812 03:13:08.233640 19250 log.cc:824] T 6a5c906b7f90447d8295f417843127dd P 948c005970b84ae88bb3b0df9b220279: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:08.234998 19251 tablet_bootstrap.cc:654] T 6a5c906b7f90447d8295f417843127dd P 19a1655d556a44179d5e0ea877a6a920: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:08.235682 19250 tablet_bootstrap.cc:492] T 6a5c906b7f90447d8295f417843127dd P 948c005970b84ae88bb3b0df9b220279: No bootstrap required, opened a new log
I20260812 03:13:08.235682 19249 tablet_bootstrap.cc:492] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401: No bootstrap required, opened a new log
I20260812 03:13:08.235878 19249 ts_tablet_manager.cc:1397] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260812 03:13:08.235878 19250 ts_tablet_manager.cc:1397] T 6a5c906b7f90447d8295f417843127dd P 948c005970b84ae88bb3b0df9b220279: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:13:08.236090 19251 log.cc:824] T 6a5c906b7f90447d8295f417843127dd P 19a1655d556a44179d5e0ea877a6a920: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:08.237644 19251 tablet_bootstrap.cc:492] T 6a5c906b7f90447d8295f417843127dd P 19a1655d556a44179d5e0ea877a6a920: No bootstrap required, opened a new log
I20260812 03:13:08.237841 19251 ts_tablet_manager.cc:1397] T 6a5c906b7f90447d8295f417843127dd P 19a1655d556a44179d5e0ea877a6a920: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:13:08.239878 19249 raft_consensus.cc:348] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65a5a8a61e4404e8f920b9cf27e1401" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36537 } } peers { permanent_uuid: "948c005970b84ae88bb3b0df9b220279" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36715 } } peers { permanent_uuid: "19a1655d556a44179d5e0ea877a6a920" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 45307 } }
I20260812 03:13:08.240166 19249 raft_consensus.cc:374] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:08.240264 19249 raft_consensus.cc:729] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65a5a8a61e4404e8f920b9cf27e1401, State: Initialized, Role: FOLLOWER
I20260812 03:13:08.240161 19250 raft_consensus.cc:348] T 6a5c906b7f90447d8295f417843127dd P 948c005970b84ae88bb3b0df9b220279 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65a5a8a61e4404e8f920b9cf27e1401" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36537 } } peers { permanent_uuid: "948c005970b84ae88bb3b0df9b220279" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36715 } } peers { permanent_uuid: "19a1655d556a44179d5e0ea877a6a920" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 45307 } }
I20260812 03:13:08.240432 19250 raft_consensus.cc:374] T 6a5c906b7f90447d8295f417843127dd P 948c005970b84ae88bb3b0df9b220279 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:08.240743 19249 consensus_queue.cc:260] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65a5a8a61e4404e8f920b9cf27e1401" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36537 } } peers { permanent_uuid: "948c005970b84ae88bb3b0df9b220279" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36715 } } peers { permanent_uuid: "19a1655d556a44179d5e0ea877a6a920" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 45307 } }
I20260812 03:13:08.240700 19251 raft_consensus.cc:348] T 6a5c906b7f90447d8295f417843127dd P 19a1655d556a44179d5e0ea877a6a920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65a5a8a61e4404e8f920b9cf27e1401" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36537 } } peers { permanent_uuid: "948c005970b84ae88bb3b0df9b220279" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36715 } } peers { permanent_uuid: "19a1655d556a44179d5e0ea877a6a920" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 45307 } }
I20260812 03:13:08.240931 19250 raft_consensus.cc:729] T 6a5c906b7f90447d8295f417843127dd P 948c005970b84ae88bb3b0df9b220279 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948c005970b84ae88bb3b0df9b220279, State: Initialized, Role: FOLLOWER
I20260812 03:13:08.240978 19251 raft_consensus.cc:374] T 6a5c906b7f90447d8295f417843127dd P 19a1655d556a44179d5e0ea877a6a920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:08.241078 19251 raft_consensus.cc:729] T 6a5c906b7f90447d8295f417843127dd P 19a1655d556a44179d5e0ea877a6a920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19a1655d556a44179d5e0ea877a6a920, State: Initialized, Role: FOLLOWER
I20260812 03:13:08.241381 19250 consensus_queue.cc:260] T 6a5c906b7f90447d8295f417843127dd P 948c005970b84ae88bb3b0df9b220279 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65a5a8a61e4404e8f920b9cf27e1401" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36537 } } peers { permanent_uuid: "948c005970b84ae88bb3b0df9b220279" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36715 } } peers { permanent_uuid: "19a1655d556a44179d5e0ea877a6a920" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 45307 } }
I20260812 03:13:08.241644 19251 consensus_queue.cc:260] T 6a5c906b7f90447d8295f417843127dd P 19a1655d556a44179d5e0ea877a6a920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65a5a8a61e4404e8f920b9cf27e1401" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36537 } } peers { permanent_uuid: "948c005970b84ae88bb3b0df9b220279" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36715 } } peers { permanent_uuid: "19a1655d556a44179d5e0ea877a6a920" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 45307 } }
I20260812 03:13:08.242072 19249 ts_tablet_manager.cc:1428] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260812 03:13:08.242188 18720 heartbeater.cc:503] Master 127.16.217.126:44505 was elected leader, sending a full tablet report...
I20260812 03:13:08.242326 19250 ts_tablet_manager.cc:1428] T 6a5c906b7f90447d8295f417843127dd P 948c005970b84ae88bb3b0df9b220279: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 03:13:08.242527 18951 heartbeater.cc:503] Master 127.16.217.126:44505 was elected leader, sending a full tablet report...
I20260812 03:13:08.242717 19251 ts_tablet_manager.cc:1428] T 6a5c906b7f90447d8295f417843127dd P 19a1655d556a44179d5e0ea877a6a920: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 03:13:08.243057 19210 heartbeater.cc:503] Master 127.16.217.126:44505 was elected leader, sending a full tablet report...
W20260812 03:13:08.407135 19211 tablet.cc:2369] T 6a5c906b7f90447d8295f417843127dd P 19a1655d556a44179d5e0ea877a6a920: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:13:08.426982 18952 tablet.cc:2369] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:13:08.430509 19260 raft_consensus.cc:482] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:08.430711 19260 raft_consensus.cc:504] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65a5a8a61e4404e8f920b9cf27e1401" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36537 } } peers { permanent_uuid: "948c005970b84ae88bb3b0df9b220279" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36715 } } peers { permanent_uuid: "19a1655d556a44179d5e0ea877a6a920" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 45307 } }
I20260812 03:13:08.431876 19260 leader_election.cc:288] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 948c005970b84ae88bb3b0df9b220279 (127.16.217.65:36715), 19a1655d556a44179d5e0ea877a6a920 (127.16.217.67:45307)
I20260812 03:13:08.435729 18955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:08.432069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:36715 (local address 127.16.217.66:51733)
0812 03:13:08.432213 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:08.432231 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:08.432333 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:08.432637 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:08.432643 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:08.432671 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:08.432810 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:08.432819 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.433644 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.433648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:08.434564 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:08.434574 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.435267 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.435271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:08.435389 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:08.435415 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 03:13:08.435489 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:13:08.435564 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":41}
I20260812 03:13:08.435735 19243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:08.432223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:51733 (local address 127.16.217.65:36715)
0812 03:13:08.432351 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:08.432355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:08.432374 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:13:08.432438 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:08.432441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:08.432494 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:08.432595 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:08.432927 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.433527 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.434693 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.435150 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.435244 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:08.435287 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:08.435518 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 03:13:08.435590 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:13:08.435858 19247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:08.432350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:60675 (local address 127.16.217.67:45307)
0812 03:13:08.432522 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:08.432526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:08.432620 (+ 94us) server_negotiation.cc:407] Connection header received
0812 03:13:08.432742 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:08.432746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:08.432796 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:08.432884 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:08.433175 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.433690 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.434844 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.435304 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.435421 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:08.435481 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:08.435669 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 03:13:08.435735 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 03:13:08.435915 19264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:08.432218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:45307 (local address 127.16.217.66:60675)
0812 03:13:08.432518 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:08.432533 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:08.432636 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:08.432922 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:08.432925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:08.432942 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:08.433060 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:08.433066 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.433802 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.433804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:08.434714 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:08.434725 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.435422 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.435425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:08.435541 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:08.435564 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:13:08.435640 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 03:13:08.435706 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":110,"threads_started":1}
I20260812 03:13:08.436456 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a5c906b7f90447d8295f417843127dd" candidate_uuid: "e65a5a8a61e4404e8f920b9cf27e1401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948c005970b84ae88bb3b0df9b220279" is_pre_election: true
I20260812 03:13:08.436458 19108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a5c906b7f90447d8295f417843127dd" candidate_uuid: "e65a5a8a61e4404e8f920b9cf27e1401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19a1655d556a44179d5e0ea877a6a920" is_pre_election: true
I20260812 03:13:08.436758 18619 raft_consensus.cc:2454] T 6a5c906b7f90447d8295f417843127dd P 948c005970b84ae88bb3b0df9b220279 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e65a5a8a61e4404e8f920b9cf27e1401 in term 0.
I20260812 03:13:08.436758 19108 raft_consensus.cc:2454] T 6a5c906b7f90447d8295f417843127dd P 19a1655d556a44179d5e0ea877a6a920 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e65a5a8a61e4404e8f920b9cf27e1401 in term 0.
I20260812 03:13:08.437235 18761 leader_election.cc:302] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 948c005970b84ae88bb3b0df9b220279, e65a5a8a61e4404e8f920b9cf27e1401; no voters:
I20260812 03:13:08.437474 19260 raft_consensus.cc:2781] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:13:08.437564 19260 raft_consensus.cc:482] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:13:08.437610 19260 raft_consensus.cc:3037] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:08.438828 19260 raft_consensus.cc:504] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65a5a8a61e4404e8f920b9cf27e1401" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36537 } } peers { permanent_uuid: "948c005970b84ae88bb3b0df9b220279" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36715 } } peers { permanent_uuid: "19a1655d556a44179d5e0ea877a6a920" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 45307 } }
I20260812 03:13:08.439229 19260 leader_election.cc:288] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401 [CANDIDATE]: Term 1 election: Requested vote from peers 948c005970b84ae88bb3b0df9b220279 (127.16.217.65:36715), 19a1655d556a44179d5e0ea877a6a920 (127.16.217.67:45307)
I20260812 03:13:08.439623 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a5c906b7f90447d8295f417843127dd" candidate_uuid: "e65a5a8a61e4404e8f920b9cf27e1401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948c005970b84ae88bb3b0df9b220279"
I20260812 03:13:08.439775 18619 raft_consensus.cc:3037] T 6a5c906b7f90447d8295f417843127dd P 948c005970b84ae88bb3b0df9b220279 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:08.439848 19108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a5c906b7f90447d8295f417843127dd" candidate_uuid: "e65a5a8a61e4404e8f920b9cf27e1401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19a1655d556a44179d5e0ea877a6a920"
I20260812 03:13:08.439972 19108 raft_consensus.cc:3037] T 6a5c906b7f90447d8295f417843127dd P 19a1655d556a44179d5e0ea877a6a920 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:08.441180 18619 raft_consensus.cc:2454] T 6a5c906b7f90447d8295f417843127dd P 948c005970b84ae88bb3b0df9b220279 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65a5a8a61e4404e8f920b9cf27e1401 in term 1.
I20260812 03:13:08.441265 19108 raft_consensus.cc:2454] T 6a5c906b7f90447d8295f417843127dd P 19a1655d556a44179d5e0ea877a6a920 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65a5a8a61e4404e8f920b9cf27e1401 in term 1.
I20260812 03:13:08.441565 18759 leader_election.cc:302] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19a1655d556a44179d5e0ea877a6a920, e65a5a8a61e4404e8f920b9cf27e1401; no voters:
I20260812 03:13:08.441807 19260 raft_consensus.cc:2781] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:08.442040 19260 raft_consensus.cc:686] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401 [term 1 LEADER]: Becoming Leader. State: Replica: e65a5a8a61e4404e8f920b9cf27e1401, State: Running, Role: LEADER
I20260812 03:13:08.442497 19260 consensus_queue.cc:237] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65a5a8a61e4404e8f920b9cf27e1401" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36537 } } peers { permanent_uuid: "948c005970b84ae88bb3b0df9b220279" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36715 } } peers { permanent_uuid: "19a1655d556a44179d5e0ea877a6a920" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 45307 } }
I20260812 03:13:08.446552 18355 catalog_manager.cc:5495] T 6a5c906b7f90447d8295f417843127dd P e65a5a8a61e4404e8f920b9cf27e1401 reported cstate change: term changed from 0 to 1, leader changed from <none> to e65a5a8a61e4404e8f920b9cf27e1401 (127.16.217.66). New cstate: current_term: 1 leader_uuid: "e65a5a8a61e4404e8f920b9cf27e1401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65a5a8a61e4404e8f920b9cf27e1401" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "948c005970b84ae88bb3b0df9b220279" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19a1655d556a44179d5e0ea877a6a920" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 45307 } health_report { overall_health: UNKNOWN } } }
W20260812 03:13:08.459107 18721 tablet.cc:2369] T 6a5c906b7f90447d8295f417843127dd P 948c005970b84ae88bb3b0df9b220279: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 03:13:08 dist-test-slave-7nm7 krb5kdc[18273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.65@KRBTEST.COM
I20260812 03:13:08.512245 19243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:08.503520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35332 (local address 127.16.217.65:36715)
0812 03:13:08.503736 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:08.503740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:08.503829 (+ 89us) server_negotiation.cc:407] Connection header received
0812 03:13:08.503946 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:08.503948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:08.503992 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:08.504062 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:08.504773 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.505283 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.506085 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.506322 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.506401 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:08.508773 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:08.508784 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:08.508801 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:08.508823 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:08.510576 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:08.510956 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:08.510961 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:08.510962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:08.511037 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:08.511369 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:08.511371 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:08.511372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:08.511597 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:08.511753 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:08.511941 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 03:13:08.512037 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
Aug 12 03:13:08 dist-test-slave-7nm7 krb5kdc[18273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504388, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:13:08 dist-test-slave-7nm7 krb5kdc[18273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504388, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.217.65@KRBTEST.COM
I20260812 03:13:08.541513 19243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:08.533102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35342 (local address 127.16.217.65:36715)
0812 03:13:08.533321 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:08.533325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:08.533346 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:13:08.533489 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:08.533492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:08.533536 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:08.533609 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:08.534295 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.534830 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.535615 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.535837 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.535921 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:08.538377 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:08.538381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:08.538383 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:08.538410 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:08.539667 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:08.540039 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:08.540042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:08.540043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:08.540127 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:08.540384 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:08.540388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:08.540389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:08.540579 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:08.540745 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:08.541074 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 03:13:08.541231 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
Aug 12 03:13:08 dist-test-slave-7nm7 krb5kdc[18273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:13:08 dist-test-slave-7nm7 krb5kdc[18273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.65@KRBTEST.COM
I20260812 03:13:08.569248 19243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:08.561145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35352 (local address 127.16.217.65:36715)
0812 03:13:08.561328 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:08.561333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:08.561426 (+ 93us) server_negotiation.cc:407] Connection header received
0812 03:13:08.561542 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:08.561545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:08.561598 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:08.561669 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:08.562371 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.562870 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.563679 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:08.563879 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:08.563958 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:08.566426 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:08.566429 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:08.566431 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:08.566454 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:08.567656 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:08.568014 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:08.568016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:08.568017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:08.568094 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:08.568414 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:08.568417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:08.568418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:08.568587 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:08.568687 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:08.568918 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 03:13:08.569014 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 03:13:08.570808 18548 tablet_service.cc:2908] Rejecting scan request for tablet 6a5c906b7f90447d8295f417843127dd: Uninitialized: clean time has not yet been initialized
I20260812 03:13:08.573052 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 18472
I20260812 03:13:08.591665 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 18726
I20260812 03:13:08.603727 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 18961
I20260812 03:13:08.626354 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 18301
2026-08-12T03:13:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2874 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:13:08 dist-test-slave-7nm7 krb5kdc[19290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:13:08 dist-test-slave-7nm7 krb5kdc[19290](info): set up 2 sockets
Aug 12 03:13:08 dist-test-slave-7nm7 krb5kdc[19290](info): commencing operation
krb5kdc: starting...
W20260812 03:13:09.942807 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:13:10 dist-test-slave-7nm7 krb5kdc[19290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:13:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:13:10.085700 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:40971
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:39955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:40971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:10.199659 19311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:10.199997 19311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:10.200074 19311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:10.204175 19311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:10.204254 19311 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 03:13:10.204288 19311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:10.204347 19311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:10.204406 19311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:10.204448 19311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:13:10.209818 19311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:39955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:40971
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:40971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:10.210994 19311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:10.212050 19311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:10.221310 19325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:10.221383 19326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:10.221310 19330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:10.221562 19311 server_base.cc:1034] running on GCE node
I20260812 03:13:10.222332 19311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:10.223634 19311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:10.224877 19311 hybrid_clock.cc:648] HybridClock initialized: now 1786504390224826 us; error 77 us; skew 500 ppm
Aug 12 03:13:10 dist-test-slave-7nm7 krb5kdc[19290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504390, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:10.227959 19311 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:13:10.229044 19311 webserver.cc:460] Webserver started at http://127.16.217.126:45451/ using document root <none> and password file <none>
I20260812 03:13:10.229651 19311 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:10.229753 19311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:10.230010 19311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:10.232221 19311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "d92e9b7db8b543f9a233403c3147b853"
format_stamp: "Formatted at 2026-08-12 03:13:10 on dist-test-slave-7nm7"
server_key: "d336eb2df1087e5b6dc0b634bde752ce"
server_key_iv: "b37f199a4b535d00bca6588ea8819e33"
server_key_version: "encryptionkey@0"
I20260812 03:13:10.232848 19311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "d92e9b7db8b543f9a233403c3147b853"
format_stamp: "Formatted at 2026-08-12 03:13:10 on dist-test-slave-7nm7"
server_key: "d336eb2df1087e5b6dc0b634bde752ce"
server_key_iv: "b37f199a4b535d00bca6588ea8819e33"
server_key_version: "encryptionkey@0"
I20260812 03:13:10.237174 19311 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:13:10.239956 19346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:10.241281 19311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:13:10.241446 19311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "d92e9b7db8b543f9a233403c3147b853"
format_stamp: "Formatted at 2026-08-12 03:13:10 on dist-test-slave-7nm7"
server_key: "d336eb2df1087e5b6dc0b634bde752ce"
server_key_iv: "b37f199a4b535d00bca6588ea8819e33"
server_key_version: "encryptionkey@0"
I20260812 03:13:10.241569 19311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:10.273524 19311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:10.276870 19311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:10.277122 19311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:10.286389 19311 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:40971
I20260812 03:13:10.286441 19462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:40971 every 8 connection(s)
I20260812 03:13:10.287372 19311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:10.290266 19463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:10.293152 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 19311
I20260812 03:13:10.293293 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:13:10.293543 17253 external_mini_cluster.cc:1428] Setting key f91cc107db22547147ea9c1e97cd78e4
I20260812 03:13:10.296362 19463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853: Bootstrap starting.
I20260812 03:13:10.299371 19463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:10.300309 19463 log.cc:824] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:10.302626 19463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853: No bootstrap required, opened a new log
Aug 12 03:13:10 dist-test-slave-7nm7 krb5kdc[19290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:10.306547 19463 raft_consensus.cc:348] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d92e9b7db8b543f9a233403c3147b853" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 40971 } }
I20260812 03:13:10.306897 19463 raft_consensus.cc:374] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:10.306980 19463 raft_consensus.cc:729] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d92e9b7db8b543f9a233403c3147b853, State: Initialized, Role: FOLLOWER
I20260812 03:13:10.307497 19463 consensus_queue.cc:260] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d92e9b7db8b543f9a233403c3147b853" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 40971 } }
I20260812 03:13:10.307672 19463 raft_consensus.cc:388] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:10.307750 19463 raft_consensus.cc:482] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:10.307857 19463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:10.308844 19463 raft_consensus.cc:504] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d92e9b7db8b543f9a233403c3147b853" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 40971 } }
I20260812 03:13:10.308986 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:10.294986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34582 (local address 127.16.217.126:40971)
0812 03:13:10.295401 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:10.295417 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:10.295454 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:13:10.296056 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:10.296081 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:10.296403 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:10.296721 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:10.297633 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:10.298729 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:10.300188 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:10.300770 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:10.300962 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:10.303822 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:10.303828 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:10.303841 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:10.303869 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:10.305849 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:10.306382 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:10.306388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:10.306392 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:10.306530 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:10.306840 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:10.306842 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:10.306843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:10.307342 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:10.307505 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:10.307870 (+ 365us) server_negotiation.cc:299] Negotiation successful
0812 03:13:10.308171 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":132,"threads_started":1}
I20260812 03:13:10.309232 19463 leader_election.cc:302] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d92e9b7db8b543f9a233403c3147b853; no voters:
I20260812 03:13:10.309578 19463 leader_election.cc:288] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:13:10.309705 19477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:10.309929 19477 raft_consensus.cc:686] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853 [term 1 LEADER]: Becoming Leader. State: Replica: d92e9b7db8b543f9a233403c3147b853, State: Running, Role: LEADER
I20260812 03:13:10.310271 19477 consensus_queue.cc:237] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d92e9b7db8b543f9a233403c3147b853" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 40971 } }
I20260812 03:13:10.310940 19463 sys_catalog.cc:564] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:10.311444 19477 sys_catalog.cc:455] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d92e9b7db8b543f9a233403c3147b853. Latest consensus state: current_term: 1 leader_uuid: "d92e9b7db8b543f9a233403c3147b853" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d92e9b7db8b543f9a233403c3147b853" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 40971 } } }
I20260812 03:13:10.311569 19477 sys_catalog.cc:458] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:10.311956 19483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:10.312279 19479 sys_catalog.cc:455] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d92e9b7db8b543f9a233403c3147b853" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d92e9b7db8b543f9a233403c3147b853" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 40971 } } }
I20260812 03:13:10.312376 19479 sys_catalog.cc:458] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:10.315109 19483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:10.321352 19483 catalog_manager.cc:1347] Generated new cluster ID: 43a09dbcd0db4203b36fa22ccb4ff84a
I20260812 03:13:10.321439 19483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:10.330394 19483 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:13:10.331385 19483 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:10.338120 19483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853: Generated new TSK 0
I20260812 03:13:10.338948 19483 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
I20260812 03:13:10.379281 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:40971
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:39955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:10.538137 19513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:10.538411 19513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:10.538455 19513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:10.542333 19513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:10.542382 19513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:10.542420 19513 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:13:10.542452 19513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:13:10.547698 19513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:39955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.217.126:40971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:10.548681 19513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:10.549651 19513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:10.559108 19524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:10.559109 19526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:10.559175 19521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:10.559238 19513 server_base.cc:1034] running on GCE node
I20260812 03:13:10.559856 19513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:10.560525 19513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:10.561726 19513 hybrid_clock.cc:648] HybridClock initialized: now 1786504390561708 us; error 31 us; skew 500 ppm
Aug 12 03:13:10 dist-test-slave-7nm7 krb5kdc[19290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504390, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:10.564657 19513 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:13:10.565608 19513 webserver.cc:460] Webserver started at http://127.16.217.65:45813/ using document root <none> and password file <none>
I20260812 03:13:10.566218 19513 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:10.566370 19513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:10.566617 19513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:10.568667 19513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654"
format_stamp: "Formatted at 2026-08-12 03:13:10 on dist-test-slave-7nm7"
server_key: "79b3c92ccbd712c2c679336f7353fd16"
server_key_iv: "7c3881a523954a555303fed3222a62e8"
server_key_version: "encryptionkey@0"
I20260812 03:13:10.569263 19513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654"
format_stamp: "Formatted at 2026-08-12 03:13:10 on dist-test-slave-7nm7"
server_key: "79b3c92ccbd712c2c679336f7353fd16"
server_key_iv: "7c3881a523954a555303fed3222a62e8"
server_key_version: "encryptionkey@0"
I20260812 03:13:10.573238 19513 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260812 03:13:10.575850 19538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:10.577037 19513 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:13:10.577171 19513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654"
format_stamp: "Formatted at 2026-08-12 03:13:10 on dist-test-slave-7nm7"
server_key: "79b3c92ccbd712c2c679336f7353fd16"
server_key_iv: "7c3881a523954a555303fed3222a62e8"
server_key_version: "encryptionkey@0"
I20260812 03:13:10.577282 19513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:10.589892 19513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:10.593031 19513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:10.593225 19513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:10.594789 19513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:10.594856 19513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:10.594923 19513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:10.594970 19513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:13:10 dist-test-slave-7nm7 krb5kdc[19290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504390, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:10.605744 19513 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:40839
I20260812 03:13:10.605903 19755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:40839 every 8 connection(s)
I20260812 03:13:10.606709 19513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:13:10.609754 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:10.595677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:56363 (local address 127.16.217.126:40971)
0812 03:13:10.595855 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:10.595862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:10.596686 (+ 824us) server_negotiation.cc:407] Connection header received
0812 03:13:10.597730 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:10.597737 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:10.597804 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:10.597876 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:10.599669 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:10.600554 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:10.602480 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:10.602804 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:10.602901 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:10.605793 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:10.605800 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:10.605803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:10.605839 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:10.607689 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:10.608228 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:10.608231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:10.608232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:10.608323 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:10.608719 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:10.608722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:10.608723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:10.608906 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:10.609006 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:10.609507 (+ 501us) server_negotiation.cc:299] Negotiation successful
0812 03:13:10.609631 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 03:13:10.610685 19553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:10.596015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:40971 (local address 127.16.217.65:56363)
0812 03:13:10.596514 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:10.596565 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:10.597469 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:10.598380 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:10.598391 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:10.598828 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:10.599507 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:10.599531 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:10.600718 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:10.600725 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:10.602328 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:10.602340 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:10.602981 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:10.602986 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:10.603198 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:10.603877 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:10.603904 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:10.605611 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:10.607813 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:10.607818 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:10.607828 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:10.608107 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:10.608425 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:10.608428 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:10.608429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:10.608595 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:10.609042 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:10.609047 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:10.609281 (+ 234us) client_negotiation.cc:769] Sending connection context
0812 03:13:10.609449 (+ 168us) client_negotiation.cc:240] Negotiation successful
0812 03:13:10.609702 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":141,"threads_started":1}
I20260812 03:13:10.612059 19756 heartbeater.cc:347] Connected to a master server at 127.16.217.126:40971
I20260812 03:13:10.612250 19756 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:10.612496 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 19513
I20260812 03:13:10.612660 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:13:10.612869 19756 heartbeater.cc:511] Master 127.16.217.126:40971 requested a full tablet report, sending...
I20260812 03:13:10.612984 17253 external_mini_cluster.cc:1428] Setting key 5399e306e1fd38e8ec5319455979d73c
I20260812 03:13:10.614888 19373 ts_manager.cc:194] Registered new tserver with Master: 4ac5d24cb3c54782b2d1ad87eb6a5654 (127.16.217.65:40839)
I20260812 03:13:10.616451 19373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:56363
WARNING: no policy specified for kudu/127.16.217.66@KRBTEST.COM; defaulting to no policy
I20260812 03:13:10.624303 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:10.614807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:55983 (local address 127.16.217.126:40971)
0812 03:13:10.614928 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:10.614933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:10.615203 (+ 270us) server_negotiation.cc:407] Connection header received
0812 03:13:10.615357 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:10.615360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:10.615406 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:10.615510 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:10.615988 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:10.616504 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:10.619345 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:10.619591 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:10.619724 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:10.620930 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:10.620934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:10.620937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:10.620964 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:10.622606 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:10.623088 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:10.623092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:10.623094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:10.623176 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:10.623435 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:10.623439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:10.623441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:10.623619 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:10.623731 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:10.624034 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 03:13:10.624145 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:13:10.624312 19772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:10.614751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:40971 (local address 127.16.217.65:55983)
0812 03:13:10.615109 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:10.615122 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:10.615255 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:10.615524 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:10.615528 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:10.615735 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:10.615836 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:10.615842 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:10.616642 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:10.616646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:10.619209 (+ 2563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:10.619218 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:10.619729 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:10.619733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:10.619828 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:10.620404 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:10.620417 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:10.620768 (+ 351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:10.622732 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:10.622735 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:10.622737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:10.622959 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:10.623272 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:10.623274 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:10.623275 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:10.623343 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:10.623722 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:10.623725 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:10.623844 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 03:13:10.623965 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 03:13:10.624114 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":108,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
I20260812 03:13:10.645346 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:40971
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:39955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:10.771508 19777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:10.771808 19777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:10.771857 19777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:10.776510 19777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:10.776592 19777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:10.776636 19777 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:13:10.776671 19777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:13:10.783202 19777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:39955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.217.126:40971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:10.784411 19777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:10.785595 19777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:10.796154 19789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:10.796159 19795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:10.796576 19792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:10.797251 19777 server_base.cc:1034] running on GCE node
I20260812 03:13:10.797669 19777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:10.798435 19777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:10.799620 19777 hybrid_clock.cc:648] HybridClock initialized: now 1786504390799584 us; error 49 us; skew 500 ppm
Aug 12 03:13:10 dist-test-slave-7nm7 krb5kdc[19290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504390, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:10.803251 19777 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:13:10.804263 19777 webserver.cc:460] Webserver started at http://127.16.217.66:44533/ using document root <none> and password file <none>
I20260812 03:13:10.804935 19777 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:10.805004 19777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:10.805382 19777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:10.807789 19777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "2fce8ee963fb49419a82de19f9cdf1d0"
format_stamp: "Formatted at 2026-08-12 03:13:10 on dist-test-slave-7nm7"
server_key: "d4db88aa25ebc5ddddccc0c5e9f5c56b"
server_key_iv: "8af9f9dd671de3bc6b7269467ea6ed55"
server_key_version: "encryptionkey@0"
I20260812 03:13:10.808535 19777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "2fce8ee963fb49419a82de19f9cdf1d0"
format_stamp: "Formatted at 2026-08-12 03:13:10 on dist-test-slave-7nm7"
server_key: "d4db88aa25ebc5ddddccc0c5e9f5c56b"
server_key_iv: "8af9f9dd671de3bc6b7269467ea6ed55"
server_key_version: "encryptionkey@0"
I20260812 03:13:10.813549 19777 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 03:13:10.817401 19806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:10.819409 19777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:10.819669 19777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "2fce8ee963fb49419a82de19f9cdf1d0"
format_stamp: "Formatted at 2026-08-12 03:13:10 on dist-test-slave-7nm7"
server_key: "d4db88aa25ebc5ddddccc0c5e9f5c56b"
server_key_iv: "8af9f9dd671de3bc6b7269467ea6ed55"
server_key_version: "encryptionkey@0"
I20260812 03:13:10.819804 19777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:10.853914 19777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:10.858870 19777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:10.859179 19777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:10.860921 19777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:10.860975 19777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:10.861047 19777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:10.861091 19777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:13:10 dist-test-slave-7nm7 krb5kdc[19290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504390, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:10.872328 19777 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:45623
I20260812 03:13:10.872829 20026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:45623 every 8 connection(s)
I20260812 03:13:10.873292 19777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:13:10.874473 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 19777
I20260812 03:13:10.874593 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:13:10.874912 17253 external_mini_cluster.cc:1428] Setting key fef1a2800fc1eff7f7e6eaefc3dfef41
I20260812 03:13:10.877029 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:10.862288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:48507 (local address 127.16.217.126:40971)
0812 03:13:10.862457 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:10.862464 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:10.863355 (+ 891us) server_negotiation.cc:407] Connection header received
0812 03:13:10.864756 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:10.864761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:10.864822 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:10.864929 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:10.866720 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:10.867347 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:10.869262 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:10.869489 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:10.869586 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:10.872729 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:10.872735 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:10.872738 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:10.872774 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:10.874601 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:10.875201 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:10.875205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:10.875206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:10.875300 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:10.875690 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:10.875693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:10.875695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:10.875895 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:10.876006 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:10.876648 (+ 642us) server_negotiation.cc:299] Negotiation successful
0812 03:13:10.876794 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 03:13:10.878015 19827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:10.862659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:40971 (local address 127.16.217.66:48507)
0812 03:13:10.863173 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:10.863228 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:10.864466 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:10.865144 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:10.865152 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:10.865696 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:10.866502 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:10.866527 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:10.867520 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:10.867526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:10.869123 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:10.869135 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:10.869676 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:10.869680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:10.869907 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:10.870676 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:10.870702 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:10.872530 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:10.874746 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:10.874751 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:10.874763 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:10.875070 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:10.875407 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:10.875410 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:10.875412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:10.875572 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:10.876006 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:10.876014 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:10.876333 (+ 319us) client_negotiation.cc:769] Sending connection context
0812 03:13:10.876555 (+ 222us) client_negotiation.cc:240] Negotiation successful
0812 03:13:10.876912 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":106,"threads_started":1}
I20260812 03:13:10.879556 20027 heartbeater.cc:347] Connected to a master server at 127.16.217.126:40971
I20260812 03:13:10.879714 20027 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:10.880379 20027 heartbeater.cc:511] Master 127.16.217.126:40971 requested a full tablet report, sending...
I20260812 03:13:10.881577 19373 ts_manager.cc:194] Registered new tserver with Master: 2fce8ee963fb49419a82de19f9cdf1d0 (127.16.217.66:45623)
I20260812 03:13:10.882371 19373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:48507
WARNING: no policy specified for kudu/127.16.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.67@KRBTEST.COM" created.
I20260812 03:13:10.891666 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:10.882291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:53907 (local address 127.16.217.126:40971)
0812 03:13:10.882414 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:10.882419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:10.883170 (+ 751us) server_negotiation.cc:407] Connection header received
0812 03:13:10.883390 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:10.883395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:10.883450 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:10.883538 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:10.884153 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:10.884815 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:10.886055 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:10.886350 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:10.886443 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:10.887943 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:10.887947 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:10.887948 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:10.887972 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:10.889465 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:10.889984 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:10.889987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:10.889989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:10.890098 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:10.890597 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:10.890606 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:10.890608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:10.890852 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:10.890971 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:10.891292 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 03:13:10.891410 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:13:10.891692 20042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:10.882509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:40971 (local address 127.16.217.66:53907)
0812 03:13:10.883063 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:10.883081 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:10.883209 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:10.883561 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:10.883567 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:10.883830 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:10.883993 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:10.884006 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:10.884957 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:10.884961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:10.885917 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:10.885933 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:10.886478 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:10.886484 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:10.886607 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:10.887260 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:10.887280 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:10.887732 (+ 452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:10.889605 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:10.889607 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:10.889609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:10.889865 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:10.890226 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:10.890229 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:10.890298 (+ 69us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:10.890467 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:10.890961 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:10.890964 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:10.891085 (+ 121us) client_negotiation.cc:769] Sending connection context
0812 03:13:10.891230 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 03:13:10.891391 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":112,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
I20260812 03:13:10.909111 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:40971
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:39955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:11.024664 20045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:11.024961 20045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:11.025009 20045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:11.029049 20045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:11.029116 20045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:11.029156 20045 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:13:11.029189 20045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:13:11.034683 20045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:39955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.217.126:40971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:11.035789 20045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:11.036850 20045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:11.047166 20057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:11.047191 20063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:11.047175 20055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:11.047535 20045 server_base.cc:1034] running on GCE node
I20260812 03:13:11.047964 20045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:11.048676 20045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:11.049885 20045 hybrid_clock.cc:648] HybridClock initialized: now 1786504391049868 us; error 27 us; skew 500 ppm
Aug 12 03:13:11 dist-test-slave-7nm7 krb5kdc[19290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504391, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:11.052745 20045 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:13:11.053833 20045 webserver.cc:460] Webserver started at http://127.16.217.67:41819/ using document root <none> and password file <none>
I20260812 03:13:11.054539 20045 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:11.054651 20045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:11.054901 20045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:11.057030 20045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "357d8f112d114394bde95ab401f87c84"
format_stamp: "Formatted at 2026-08-12 03:13:11 on dist-test-slave-7nm7"
server_key: "1cc9ae6b0efb2f7a1d56874bdc851f41"
server_key_iv: "f66e53d43a5715162413c9c1103142cf"
server_key_version: "encryptionkey@0"
I20260812 03:13:11.057632 20045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "357d8f112d114394bde95ab401f87c84"
format_stamp: "Formatted at 2026-08-12 03:13:11 on dist-test-slave-7nm7"
server_key: "1cc9ae6b0efb2f7a1d56874bdc851f41"
server_key_iv: "f66e53d43a5715162413c9c1103142cf"
server_key_version: "encryptionkey@0"
I20260812 03:13:11.061710 20045 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:13:11.064409 20074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:11.065718 20045 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 03:13:11.065864 20045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "357d8f112d114394bde95ab401f87c84"
format_stamp: "Formatted at 2026-08-12 03:13:11 on dist-test-slave-7nm7"
server_key: "1cc9ae6b0efb2f7a1d56874bdc851f41"
server_key_iv: "f66e53d43a5715162413c9c1103142cf"
server_key_version: "encryptionkey@0"
I20260812 03:13:11.065985 20045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:11.087488 20045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:11.090991 20045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:11.091216 20045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:11.092790 20045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:11.092870 20045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:11.092947 20045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:11.092993 20045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:13:11 dist-test-slave-7nm7 krb5kdc[19290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504391, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:11.104322 20045 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:35251
I20260812 03:13:11.105216 20307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:35251 every 8 connection(s)
I20260812 03:13:11.105495 20045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:13:11.106310 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 20045
I20260812 03:13:11.106444 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:13:11.106788 17253 external_mini_cluster.cc:1428] Setting key 36e3844124d10550377cad61f6af356b
I20260812 03:13:11.107744 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.093543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:34137 (local address 127.16.217.126:40971)
0812 03:13:11.093712 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:11.093719 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:11.095194 (+ 1475us) server_negotiation.cc:407] Connection header received
0812 03:13:11.096370 (+ 1176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:11.096374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:11.096431 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:11.096540 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:11.097828 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.098475 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.099981 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.100206 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.100315 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.103107 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:11.103111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:11.103114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:11.103141 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:11.104984 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:11.106011 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:11.106014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:11.106016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:11.106155 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:11.106617 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:11.106621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:11.106622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:11.106863 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:11.106972 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:11.107452 (+ 480us) server_negotiation.cc:299] Negotiation successful
0812 03:13:11.107572 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:13:11.108614 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.094479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:40971 (local address 127.16.217.67:34137)
0812 03:13:11.095027 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:11.095084 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:11.096160 (+ 1076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:11.096709 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:11.096716 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:11.097221 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:11.097681 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.097694 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.098596 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.098600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.099846 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.099856 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.100376 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.100380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.100553 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.101170 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:11.101187 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:11.102937 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:11.105299 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:11.105340 (+ 41us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:11.105362 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:11.105862 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:11.106270 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:11.106273 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:11.106275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:11.106447 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:11.106974 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:11.106978 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:11.107207 (+ 229us) client_negotiation.cc:769] Sending connection context
0812 03:13:11.107388 (+ 181us) client_negotiation.cc:240] Negotiation successful
0812 03:13:11.107703 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":136,"threads_started":1}
I20260812 03:13:11.109778 20309 heartbeater.cc:347] Connected to a master server at 127.16.217.126:40971
I20260812 03:13:11.109918 20309 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:11.110487 20309 heartbeater.cc:511] Master 127.16.217.126:40971 requested a full tablet report, sending...
I20260812 03:13:11.111645 19373 ts_manager.cc:194] Registered new tserver with Master: 357d8f112d114394bde95ab401f87c84 (127.16.217.67:35251)
I20260812 03:13:11.112418 19373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:34137
I20260812 03:13:11.119433 20322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.112261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:40971 (local address 127.16.217.67:56965)
0812 03:13:11.112535 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:11.112552 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:11.112683 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:11.113032 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:11.113035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:11.113256 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:11.113366 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.113374 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.114144 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.114148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.114821 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.114831 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.115289 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.115292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.115361 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.115877 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:11.115899 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:11.116205 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:11.117975 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:11.117978 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:11.117979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:11.118195 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:11.118570 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:11.118572 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:11.118574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:11.118661 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:11.118999 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:11.119001 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:11.119100 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 03:13:11.119185 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 03:13:11.119295 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":95,"threads_started":1}
I20260812 03:13:11.119565 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.112347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:56965 (local address 127.16.217.126:40971)
0812 03:13:11.112499 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:11.112523 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:11.112618 (+ 95us) server_negotiation.cc:407] Connection header received
0812 03:13:11.112792 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:11.112795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:11.112837 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:11.112883 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:11.113488 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.114001 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.114947 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.115175 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.115268 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.116358 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:11.116361 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:11.116363 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:11.116384 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:11.117871 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:11.118382 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:11.118385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:11.118386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:11.118463 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:11.118747 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:11.118750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:11.118751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:11.118915 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:11.118990 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:11.119282 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 03:13:11.119393 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 03:13:11.121040 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:13:11 dist-test-slave-7nm7 krb5kdc[19290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504391, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:13:11 dist-test-slave-7nm7 krb5kdc[19290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504391, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:11.149871 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.140745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34588 (local address 127.16.217.126:40971)
0812 03:13:11.140895 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:11.140899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:11.140913 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:13:11.140957 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:11.140959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:11.141001 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:11.141090 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:11.141808 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.142411 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.143263 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.143475 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.143565 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.146418 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:11.146421 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:11.146423 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:11.146450 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:11.147945 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:11.148476 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:11.148479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:11.148481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:11.148562 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:11.148855 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:11.148857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:11.148859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:11.149045 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:11.149301 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:11.149571 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 03:13:11.149692 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 03:13:11.153741 19373 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:13:11.157179 19373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:13:11.175447 20359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.168157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42288 (local address 127.16.217.67:35251)
0812 03:13:11.169311 (+ 1154us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:11.169325 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:11.169349 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:13:11.169430 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:11.169435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:11.169683 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:11.169839 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:11.170192 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.171534 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.173608 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.174623 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.174750 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.175102 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:11.175176 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 03:13:11.175286 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1030,"thread_start_us":126,"threads_started":1}
I20260812 03:13:11.175437 20357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.167933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:35251 (local address 127.0.0.1:42288)
0812 03:13:11.168795 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:11.168848 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:11.168993 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:11.169850 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:11.169857 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:11.169888 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:11.170051 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.170063 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.171669 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.171673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.173364 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.173375 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.174771 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.174776 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.174904 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.174954 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 03:13:11.175111 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 03:13:11.175245 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":135,"spinlock_wait_cycles":6912,"thread_start_us":101,"threads_started":1}
I20260812 03:13:11.175498 20358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.168111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:45623 (local address 127.0.0.1:44600)
0812 03:13:11.168793 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:11.168847 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:11.168993 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:11.170204 (+ 1211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:11.170207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:11.170268 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:11.170464 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.170471 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.171816 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.171819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.174018 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.174027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.175154 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.175157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.175235 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.175255 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 03:13:11.175325 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 03:13:11.175381 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":121,"spinlock_wait_cycles":14720,"thread_start_us":112,"threads_started":1}
I20260812 03:13:11.176110 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.168682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:40839 (local address 127.0.0.1:40640)
0812 03:13:11.169282 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:11.169299 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:11.169389 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:11.170205 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:11.170208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:11.170267 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:11.170464 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.170471 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.172058 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.172061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.173361 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.173374 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.175676 (+ 2302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.175682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.175779 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.175809 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 03:13:11.175896 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 03:13:11.175970 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":155,"spinlock_wait_cycles":70144,"thread_start_us":90,"threads_started":1}
I20260812 03:13:11.176455 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.169195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40640 (local address 127.16.217.65:40839)
0812 03:13:11.169547 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:11.169553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:11.169572 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:13:11.169655 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:11.169672 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:11.169839 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:11.169973 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:11.170632 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.171928 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.173660 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.174623 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.174750 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.174908 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:11.175947 (+ 1039us) server_negotiation.cc:299] Negotiation successful
0812 03:13:11.176046 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":102,"threads_started":1}
I20260812 03:13:11.176710 20362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.168607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44600 (local address 127.16.217.66:45623)
0812 03:13:11.169647 (+ 1040us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:11.169659 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:11.169679 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:13:11.169740 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:11.169744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:11.169944 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:11.170121 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:11.170628 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.171692 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.174148 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.174830 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.174934 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.175089 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:11.176442 (+ 1353us) server_negotiation.cc:299] Negotiation successful
0812 03:13:11.176561 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":942,"thread_start_us":184,"threads_started":1}
I20260812 03:13:11.178861 20179 tablet_service.cc:1467] Processing CreateTablet for tablet a2b488b01eba4bfb8891cb48db455cd2 (DEFAULT_TABLE table=test-table [id=eb3b5e004a7e480dac20099f30f3b2a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:11.179148 19636 tablet_service.cc:1467] Processing CreateTablet for tablet a2b488b01eba4bfb8891cb48db455cd2 (DEFAULT_TABLE table=test-table [id=eb3b5e004a7e480dac20099f30f3b2a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:11.179148 19905 tablet_service.cc:1467] Processing CreateTablet for tablet a2b488b01eba4bfb8891cb48db455cd2 (DEFAULT_TABLE table=test-table [id=eb3b5e004a7e480dac20099f30f3b2a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:11.180015 20179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2b488b01eba4bfb8891cb48db455cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:11.180147 19636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2b488b01eba4bfb8891cb48db455cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:11.180147 19905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2b488b01eba4bfb8891cb48db455cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:11.187220 20367 tablet_bootstrap.cc:492] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84: Bootstrap starting.
I20260812 03:13:11.188666 20368 tablet_bootstrap.cc:492] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654: Bootstrap starting.
I20260812 03:13:11.190385 20367 tablet_bootstrap.cc:654] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:11.190644 20368 tablet_bootstrap.cc:654] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:11.191200 20367 log.cc:824] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:11.191351 20368 log.cc:824] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:11.191969 20369 tablet_bootstrap.cc:492] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0: Bootstrap starting.
I20260812 03:13:11.193522 20367 tablet_bootstrap.cc:492] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84: No bootstrap required, opened a new log
I20260812 03:13:11.193753 20367 ts_tablet_manager.cc:1397] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260812 03:13:11.193892 20369 tablet_bootstrap.cc:654] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:11.194770 20369 log.cc:824] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:11.196615 20368 tablet_bootstrap.cc:492] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654: No bootstrap required, opened a new log
I20260812 03:13:11.196837 20368 ts_tablet_manager.cc:1397] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260812 03:13:11.197329 20369 tablet_bootstrap.cc:492] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0: No bootstrap required, opened a new log
I20260812 03:13:11.197506 20369 ts_tablet_manager.cc:1397] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260812 03:13:11.198040 20367 raft_consensus.cc:348] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } }
I20260812 03:13:11.198350 20367 raft_consensus.cc:374] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:11.198407 20367 raft_consensus.cc:729] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 357d8f112d114394bde95ab401f87c84, State: Initialized, Role: FOLLOWER
I20260812 03:13:11.198879 20367 consensus_queue.cc:260] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } }
I20260812 03:13:11.199589 20309 heartbeater.cc:503] Master 127.16.217.126:40971 was elected leader, sending a full tablet report...
I20260812 03:13:11.201006 20367 ts_tablet_manager.cc:1428] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84: Time spent starting tablet: real 0.007s user 0.006s sys 0.002s
I20260812 03:13:11.201615 20368 raft_consensus.cc:348] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } }
I20260812 03:13:11.201615 20369 raft_consensus.cc:348] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } }
I20260812 03:13:11.201884 20369 raft_consensus.cc:374] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:11.201884 20368 raft_consensus.cc:374] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:11.201936 20368 raft_consensus.cc:729] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ac5d24cb3c54782b2d1ad87eb6a5654, State: Initialized, Role: FOLLOWER
I20260812 03:13:11.201946 20369 raft_consensus.cc:729] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fce8ee963fb49419a82de19f9cdf1d0, State: Initialized, Role: FOLLOWER
I20260812 03:13:11.202425 20368 consensus_queue.cc:260] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } }
I20260812 03:13:11.202425 20369 consensus_queue.cc:260] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } }
I20260812 03:13:11.203382 20369 ts_tablet_manager.cc:1428] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 03:13:11.203382 20368 ts_tablet_manager.cc:1428] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260812 03:13:11.203725 19756 heartbeater.cc:503] Master 127.16.217.126:40971 was elected leader, sending a full tablet report...
I20260812 03:13:11.203768 20027 heartbeater.cc:503] Master 127.16.217.126:40971 was elected leader, sending a full tablet report...
W20260812 03:13:11.356487 20310 tablet.cc:2369] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:13:11.358282 19757 tablet.cc:2369] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:13:11.375928 20028 tablet.cc:2369] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:13:11.572432 20375 raft_consensus.cc:482] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:11.572696 20375 raft_consensus.cc:504] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } }
I20260812 03:13:11.573912 20375 leader_election.cc:288] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2fce8ee963fb49419a82de19f9cdf1d0 (127.16.217.66:45623), 4ac5d24cb3c54782b2d1ad87eb6a5654 (127.16.217.65:40839)
I20260812 03:13:11.578403 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.574346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:45623 (local address 127.16.217.67:36403)
0812 03:13:11.574508 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:11.574526 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:11.574649 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:11.574982 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:11.574986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:11.575015 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:11.575169 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.575179 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.576108 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.576111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.577016 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.577025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.577953 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.577956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.578045 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.578069 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:13:11.578141 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 03:13:11.578199 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 03:13:11.578418 20362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.574439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:36403 (local address 127.16.217.66:45623)
0812 03:13:11.574598 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:11.574603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:11.574620 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:13:11.574772 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:11.574775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:11.574851 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:11.574929 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:11.575302 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.575964 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.577155 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.577776 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.577910 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.577965 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:11.578183 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 03:13:11.578301 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 03:13:11.579133 19927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2b488b01eba4bfb8891cb48db455cd2" candidate_uuid: "357d8f112d114394bde95ab401f87c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" is_pre_election: true
I20260812 03:13:11.579475 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.574857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:40701 (local address 127.16.217.65:40839)
0812 03:13:11.574996 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:11.575001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:11.575097 (+ 96us) server_negotiation.cc:407] Connection header received
0812 03:13:11.575204 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:11.575210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:11.575275 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:11.575364 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:11.575707 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.576318 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.578297 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.578920 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.579046 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.579102 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:11.579262 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 03:13:11.579361 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:13:11.579572 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.574691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:40839 (local address 127.16.217.67:40701)
0812 03:13:11.574997 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:11.575012 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:11.575119 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:11.575391 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:11.575393 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:11.575411 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:11.575537 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.575543 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.576450 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.576453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.578152 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.578165 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.579061 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.579064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.579142 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.579160 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 03:13:11.579226 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 03:13:11.579401 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":93,"threads_started":1}
I20260812 03:13:11.579522 19927 raft_consensus.cc:2454] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 357d8f112d114394bde95ab401f87c84 in term 0.
I20260812 03:13:11.580044 20081 leader_election.cc:302] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fce8ee963fb49419a82de19f9cdf1d0, 357d8f112d114394bde95ab401f87c84; no voters:
I20260812 03:13:11.580210 19679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2b488b01eba4bfb8891cb48db455cd2" candidate_uuid: "357d8f112d114394bde95ab401f87c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" is_pre_election: true
I20260812 03:13:11.580366 20375 raft_consensus.cc:2781] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:13:11.580468 20375 raft_consensus.cc:482] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:13:11.580467 19679 raft_consensus.cc:2454] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 357d8f112d114394bde95ab401f87c84 in term 0.
I20260812 03:13:11.580529 20375 raft_consensus.cc:3037] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:11.581791 20375 raft_consensus.cc:504] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } }
I20260812 03:13:11.582190 20375 leader_election.cc:288] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [CANDIDATE]: Term 1 election: Requested vote from peers 2fce8ee963fb49419a82de19f9cdf1d0 (127.16.217.66:45623), 4ac5d24cb3c54782b2d1ad87eb6a5654 (127.16.217.65:40839)
I20260812 03:13:11.582564 19927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2b488b01eba4bfb8891cb48db455cd2" candidate_uuid: "357d8f112d114394bde95ab401f87c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fce8ee963fb49419a82de19f9cdf1d0"
I20260812 03:13:11.582664 19679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2b488b01eba4bfb8891cb48db455cd2" candidate_uuid: "357d8f112d114394bde95ab401f87c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654"
I20260812 03:13:11.582685 19927 raft_consensus.cc:3037] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:11.582796 19679 raft_consensus.cc:3037] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:11.583710 19927 raft_consensus.cc:2454] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 357d8f112d114394bde95ab401f87c84 in term 1.
I20260812 03:13:11.584033 20081 leader_election.cc:302] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fce8ee963fb49419a82de19f9cdf1d0, 357d8f112d114394bde95ab401f87c84; no voters:
I20260812 03:13:11.584199 20375 raft_consensus.cc:2781] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:11.584172 19679 raft_consensus.cc:2454] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 357d8f112d114394bde95ab401f87c84 in term 1.
I20260812 03:13:11.584438 20375 raft_consensus.cc:686] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [term 1 LEADER]: Becoming Leader. State: Replica: 357d8f112d114394bde95ab401f87c84, State: Running, Role: LEADER
I20260812 03:13:11.584812 20375 consensus_queue.cc:237] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } }
I20260812 03:13:11.588081 19372 catalog_manager.cc:5495] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 357d8f112d114394bde95ab401f87c84 (127.16.217.67). New cstate: current_term: 1 leader_uuid: "357d8f112d114394bde95ab401f87c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } health_report { overall_health: UNKNOWN } } }
I20260812 03:13:11.637954 20359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.634184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42294 (local address 127.16.217.67:35251)
0812 03:13:11.634438 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:11.634445 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:11.634474 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:13:11.634560 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:11.634562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:11.634617 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:11.634720 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:11.634998 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.635519 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.636408 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.636638 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.636752 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.637096 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:11.637495 (+ 399us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:11.637627 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:11.637738 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 03:13:11.637797 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260812 03:13:11.643846 19679 raft_consensus.cc:1261] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 1 FOLLOWER]: Refusing update from remote peer 357d8f112d114394bde95ab401f87c84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:11.643915 19927 raft_consensus.cc:1261] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0 [term 1 FOLLOWER]: Refusing update from remote peer 357d8f112d114394bde95ab401f87c84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:11.644718 20375 consensus_queue.cc:1035] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:11.644968 20379 consensus_queue.cc:1035] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:11.653887 20385 mvcc.cc:205] Tried to move back new op lower bound from 7317521988165591040 to 7317521987934171136. Current Snapshot: MvccSnapshot[applied={T|T < 7317521988165591040}]
I20260812 03:13:11.657179 20386 mvcc.cc:205] Tried to move back new op lower bound from 7317521988165591040 to 7317521987934171136. Current Snapshot: MvccSnapshot[applied={T|T < 7317521988165591040}]
I20260812 03:13:11.658774 20387 mvcc.cc:205] Tried to move back new op lower bound from 7317521988165591040 to 7317521987934171136. Current Snapshot: MvccSnapshot[applied={T|T < 7317521988165591040 or (T in {7317521988165591040})}]
I20260812 03:13:11.665936 19373 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:11.666098 19373 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:11.668723 19373 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853: Sending DeleteTablet for 3 replicas of tablet a2b488b01eba4bfb8891cb48db455cd2
I20260812 03:13:11.669600 20179 tablet_service.cc:1514] Processing DeleteTablet for tablet a2b488b01eba4bfb8891cb48db455cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:11 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:42288
I20260812 03:13:11.669651 19905 tablet_service.cc:1514] Processing DeleteTablet for tablet a2b488b01eba4bfb8891cb48db455cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:11 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:44600
I20260812 03:13:11.669795 19636 tablet_service.cc:1514] Processing DeleteTablet for tablet a2b488b01eba4bfb8891cb48db455cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:11 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:40640
I20260812 03:13:11.670630 20408 tablet_replica.cc:330] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654: stopping tablet replica
I20260812 03:13:11.670612 20404 tablet_replica.cc:330] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0: stopping tablet replica
I20260812 03:13:11.670689 19373 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 03:13:11.671000 20404 raft_consensus.cc:2229] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260812 03:13:11.671119 19373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:13:11.671339 20404 raft_consensus.cc:2258] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:11.671373 20408 raft_consensus.cc:2229] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:11.671784 20408 raft_consensus.cc:2258] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:11.672897 20404 ts_tablet_manager.cc:1905] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:11.673183 20408 ts_tablet_manager.cc:1905] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:11.674346 20400 tablet_replica.cc:330] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84: stopping tablet replica
I20260812 03:13:11.674654 20400 raft_consensus.cc:2229] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:13:11.675028 20400 raft_consensus.cc:2258] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:11.676505 20408 ts_tablet_manager.cc:1918] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:13:11.676678 20179 tablet_service.cc:1467] Processing CreateTablet for tablet bd6f8fd319664616a90f5e92034c0b6e (DEFAULT_TABLE table=test-table [id=917bf53edb4d4211bddf062fc4dad3df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:11.676918 20408 log.cc:1196] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/wal/wals/a2b488b01eba4bfb8891cb48db455cd2
I20260812 03:13:11.676965 20179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd6f8fd319664616a90f5e92034c0b6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:11.677393 20408 ts_tablet_manager.cc:1939] T a2b488b01eba4bfb8891cb48db455cd2 P 4ac5d24cb3c54782b2d1ad87eb6a5654: Deleting consensus metadata
I20260812 03:13:11.677839 20400 ts_tablet_manager.cc:1905] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:11.679450 20367 tablet_bootstrap.cc:492] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84: Bootstrap starting.
I20260812 03:13:11.679654 19354 catalog_manager.cc:4841] TS 4ac5d24cb3c54782b2d1ad87eb6a5654 (127.16.217.65:40839): tablet a2b488b01eba4bfb8891cb48db455cd2 (table test-table [id=eb3b5e004a7e480dac20099f30f3b2a0]) successfully deleted
I20260812 03:13:11.680163 20404 ts_tablet_manager.cc:1918] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:13:11.680261 20404 log.cc:1196] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/a2b488b01eba4bfb8891cb48db455cd2
I20260812 03:13:11.680477 20367 tablet_bootstrap.cc:654] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:11.680480 19905 tablet_service.cc:1467] Processing CreateTablet for tablet bd6f8fd319664616a90f5e92034c0b6e (DEFAULT_TABLE table=test-table [id=917bf53edb4d4211bddf062fc4dad3df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:11.680595 20404 ts_tablet_manager.cc:1939] T a2b488b01eba4bfb8891cb48db455cd2 P 2fce8ee963fb49419a82de19f9cdf1d0: Deleting consensus metadata
I20260812 03:13:11.680724 19905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd6f8fd319664616a90f5e92034c0b6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:11.682058 20367 tablet_bootstrap.cc:492] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84: No bootstrap required, opened a new log
I20260812 03:13:11.682165 20367 ts_tablet_manager.cc:1397] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 03:13:11.682389 20400 ts_tablet_manager.cc:1918] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:13:11.682466 20400 log.cc:1196] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/a2b488b01eba4bfb8891cb48db455cd2
I20260812 03:13:11.682798 20400 ts_tablet_manager.cc:1939] T a2b488b01eba4bfb8891cb48db455cd2 P 357d8f112d114394bde95ab401f87c84: Deleting consensus metadata
I20260812 03:13:11.682713 20367 raft_consensus.cc:348] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } }
I20260812 03:13:11.682855 20367 raft_consensus.cc:374] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:11.682896 20367 raft_consensus.cc:729] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 357d8f112d114394bde95ab401f87c84, State: Initialized, Role: FOLLOWER
I20260812 03:13:11.683008 20369 tablet_bootstrap.cc:492] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0: Bootstrap starting.
I20260812 03:13:11.683018 20367 consensus_queue.cc:260] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } }
I20260812 03:13:11.683318 20367 ts_tablet_manager.cc:1428] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:11.683918 20369 tablet_bootstrap.cc:654] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:11.685711 19636 tablet_service.cc:1467] Processing CreateTablet for tablet bd6f8fd319664616a90f5e92034c0b6e (DEFAULT_TABLE table=test-table [id=917bf53edb4d4211bddf062fc4dad3df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:11.685747 19348 catalog_manager.cc:4841] TS 357d8f112d114394bde95ab401f87c84 (127.16.217.67:35251): tablet a2b488b01eba4bfb8891cb48db455cd2 (table test-table [id=eb3b5e004a7e480dac20099f30f3b2a0]) successfully deleted
I20260812 03:13:11.685997 19636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd6f8fd319664616a90f5e92034c0b6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:11.686743 19354 catalog_manager.cc:4841] TS 2fce8ee963fb49419a82de19f9cdf1d0 (127.16.217.66:45623): tablet a2b488b01eba4bfb8891cb48db455cd2 (table test-table [id=eb3b5e004a7e480dac20099f30f3b2a0]) successfully deleted
I20260812 03:13:11.687616 20369 tablet_bootstrap.cc:492] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0: No bootstrap required, opened a new log
I20260812 03:13:11.687706 20369 ts_tablet_manager.cc:1397] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 03:13:11.688158 20369 raft_consensus.cc:348] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } }
I20260812 03:13:11.688265 20369 raft_consensus.cc:374] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:11.688295 20369 raft_consensus.cc:729] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fce8ee963fb49419a82de19f9cdf1d0, State: Initialized, Role: FOLLOWER
I20260812 03:13:11.688352 20368 tablet_bootstrap.cc:492] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654: Bootstrap starting.
I20260812 03:13:11.688408 20369 consensus_queue.cc:260] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } }
I20260812 03:13:11.688653 20369 ts_tablet_manager.cc:1428] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:11.689751 20368 tablet_bootstrap.cc:654] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:11.690990 20368 tablet_bootstrap.cc:492] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654: No bootstrap required, opened a new log
I20260812 03:13:11.691063 20368 ts_tablet_manager.cc:1397] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 03:13:11.691461 20368 raft_consensus.cc:348] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } }
I20260812 03:13:11.691552 20368 raft_consensus.cc:374] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:11.691574 20368 raft_consensus.cc:729] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ac5d24cb3c54782b2d1ad87eb6a5654, State: Initialized, Role: FOLLOWER
I20260812 03:13:11.691769 20368 consensus_queue.cc:260] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } }
I20260812 03:13:11.692041 20368 ts_tablet_manager.cc:1428] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:13:11.746352 20377 raft_consensus.cc:482] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:11.746505 20377 raft_consensus.cc:504] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } }
I20260812 03:13:11.747735 20377 leader_election.cc:288] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ac5d24cb3c54782b2d1ad87eb6a5654 (127.16.217.65:40839), 357d8f112d114394bde95ab401f87c84 (127.16.217.67:35251)
I20260812 03:13:11.752903 20419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.748006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:40839 (local address 127.16.217.66:48557)
0812 03:13:11.748567 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:11.748581 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:11.748733 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:11.749068 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:11.749071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:11.749100 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:11.749227 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.749233 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.750578 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.750581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.751457 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.751467 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.752398 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.752402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.752508 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.752572 (+ 64us) client_negotiation.cc:769] Sending connection context
0812 03:13:11.752654 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:13:11.752727 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":23,"thread_start_us":120,"threads_started":1}
I20260812 03:13:11.752934 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.748342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:48557 (local address 127.16.217.65:40839)
0812 03:13:11.748511 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:11.748515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:11.748658 (+ 143us) server_negotiation.cc:407] Connection header received
0812 03:13:11.748848 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:11.748851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:11.748895 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:11.748941 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:11.749354 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.750178 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.751615 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.752244 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.752383 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.752441 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:11.752709 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 03:13:11.752806 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:13:11.752978 20359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.748099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:57851 (local address 127.16.217.67:35251)
0812 03:13:11.748243 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:11.748248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:11.748419 (+ 171us) server_negotiation.cc:407] Connection header received
0812 03:13:11.748648 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:11.748653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:11.748707 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:11.748798 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:11.749083 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.749853 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.751672 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.752268 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.752387 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.752441 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:11.752808 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 03:13:11.752876 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:13:11.753384 20418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.747995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:35251 (local address 127.16.217.66:57851)
0812 03:13:11.748323 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:11.748338 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:11.748509 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:11.748813 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:11.748817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:11.748834 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:11.748959 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.748965 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.749999 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.750003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.751563 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.751574 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.752513 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.752516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.752650 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.752682 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 03:13:11.753104 (+ 422us) client_negotiation.cc:240] Negotiation successful
0812 03:13:11.753207 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":2,"thread_start_us":318,"threads_started":1}
I20260812 03:13:11.753613 19679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd6f8fd319664616a90f5e92034c0b6e" candidate_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" is_pre_election: true
I20260812 03:13:11.753805 19679 raft_consensus.cc:2454] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fce8ee963fb49419a82de19f9cdf1d0 in term 0.
I20260812 03:13:11.753954 20223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd6f8fd319664616a90f5e92034c0b6e" candidate_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "357d8f112d114394bde95ab401f87c84" is_pre_election: true
I20260812 03:13:11.754215 20223 raft_consensus.cc:2454] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fce8ee963fb49419a82de19f9cdf1d0 in term 0.
I20260812 03:13:11.754213 19819 leader_election.cc:302] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fce8ee963fb49419a82de19f9cdf1d0, 4ac5d24cb3c54782b2d1ad87eb6a5654; no voters:
I20260812 03:13:11.754519 20377 raft_consensus.cc:2781] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:13:11.754585 20377 raft_consensus.cc:482] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:13:11.754629 20377 raft_consensus.cc:3037] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:11.755662 20377 raft_consensus.cc:504] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } }
I20260812 03:13:11.756088 20377 leader_election.cc:288] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [CANDIDATE]: Term 1 election: Requested vote from peers 4ac5d24cb3c54782b2d1ad87eb6a5654 (127.16.217.65:40839), 357d8f112d114394bde95ab401f87c84 (127.16.217.67:35251)
I20260812 03:13:11.756433 19679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd6f8fd319664616a90f5e92034c0b6e" candidate_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654"
I20260812 03:13:11.756470 20223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd6f8fd319664616a90f5e92034c0b6e" candidate_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "357d8f112d114394bde95ab401f87c84"
I20260812 03:13:11.756551 19679 raft_consensus.cc:3037] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:11.756599 20223 raft_consensus.cc:3037] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:11.757637 20223 raft_consensus.cc:2454] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fce8ee963fb49419a82de19f9cdf1d0 in term 1.
I20260812 03:13:11.757714 19679 raft_consensus.cc:2454] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fce8ee963fb49419a82de19f9cdf1d0 in term 1.
I20260812 03:13:11.757977 19816 leader_election.cc:302] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fce8ee963fb49419a82de19f9cdf1d0, 357d8f112d114394bde95ab401f87c84; no voters:
I20260812 03:13:11.758140 20377 raft_consensus.cc:2781] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:11.758409 20377 raft_consensus.cc:686] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [term 1 LEADER]: Becoming Leader. State: Replica: 2fce8ee963fb49419a82de19f9cdf1d0, State: Running, Role: LEADER
I20260812 03:13:11.758872 20377 consensus_queue.cc:237] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } }
I20260812 03:13:11.761468 19372 catalog_manager.cc:5495] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fce8ee963fb49419a82de19f9cdf1d0 (127.16.217.66). New cstate: current_term: 1 leader_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } health_report { overall_health: UNKNOWN } } }
Aug 12 03:13:11 dist-test-slave-7nm7 krb5kdc[19290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504390, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:11.806761 20441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.796167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:40971 (local address 127.0.0.1:34602)
0812 03:13:11.796593 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:11.796610 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:11.796749 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:11.797070 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:11.797073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:11.797676 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:11.797811 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.797820 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.799090 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.799094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.799938 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.799951 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.800510 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.800515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.800613 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.801299 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:11.801315 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:11.803091 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:11.804944 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:11.804955 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:11.804963 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:11.805206 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:11.805494 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:11.805498 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:11.805500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:11.805586 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:11.806152 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:11.806158 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:11.806343 (+ 185us) client_negotiation.cc:769] Sending connection context
0812 03:13:11.806468 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 03:13:11.806607 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":147,"threads_started":1}
I20260812 03:13:11.806906 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.796295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34602 (local address 127.16.217.126:40971)
0812 03:13:11.796573 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:11.796581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:11.796719 (+ 138us) server_negotiation.cc:407] Connection header received
0812 03:13:11.796877 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:11.796879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:11.796923 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:11.797003 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:11.797962 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.798943 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.800121 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.800373 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.800463 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.803253 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:11.803256 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:11.803258 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:11.803278 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:11.804791 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:11.805311 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:11.805316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:11.805318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:11.805401 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:11.805733 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:11.805737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:11.805739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:11.805998 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:11.806135 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:11.806558 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 03:13:11.806700 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":96,"threads_started":1}
I20260812 03:13:11.812194 19372 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:34602:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 03:13:11.813304 19372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:13:11.820542 19636 tablet_service.cc:1467] Processing CreateTablet for tablet 6efef2eb26b248028163ad874a0cca31 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ed6e18d703134f4b9b9db990daef2751]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:13:11.820982 19636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6efef2eb26b248028163ad874a0cca31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:11.824528 20368 tablet_bootstrap.cc:492] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654: Bootstrap starting.
I20260812 03:13:11.824936 20179 tablet_service.cc:1467] Processing CreateTablet for tablet 6efef2eb26b248028163ad874a0cca31 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ed6e18d703134f4b9b9db990daef2751]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:13:11.825351 20179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6efef2eb26b248028163ad874a0cca31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:11.825743 20368 tablet_bootstrap.cc:654] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:11.828598 20367 tablet_bootstrap.cc:492] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84: Bootstrap starting.
I20260812 03:13:11.828781 19905 tablet_service.cc:1467] Processing CreateTablet for tablet 6efef2eb26b248028163ad874a0cca31 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ed6e18d703134f4b9b9db990daef2751]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:13:11.829208 19905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6efef2eb26b248028163ad874a0cca31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:11.830473 20367 tablet_bootstrap.cc:654] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:11.832062 20367 tablet_bootstrap.cc:492] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84: No bootstrap required, opened a new log
I20260812 03:13:11.832182 20367 ts_tablet_manager.cc:1397] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260812 03:13:11.832593 20367 raft_consensus.cc:348] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } }
I20260812 03:13:11.832713 20369 tablet_bootstrap.cc:492] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0: Bootstrap starting.
I20260812 03:13:11.832729 20367 raft_consensus.cc:374] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:11.832768 20367 raft_consensus.cc:729] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 357d8f112d114394bde95ab401f87c84, State: Initialized, Role: FOLLOWER
I20260812 03:13:11.832919 20367 consensus_queue.cc:260] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } }
I20260812 03:13:11.833264 20391 tablet_replica.cc:439] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } }
I20260812 03:13:11.833411 20391 tablet_replica.cc:442] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:11.833702 20369 tablet_bootstrap.cc:654] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:11.835039 20369 tablet_bootstrap.cc:492] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0: No bootstrap required, opened a new log
I20260812 03:13:11.835148 20369 ts_tablet_manager.cc:1397] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:13:11.835214 20367 ts_tablet_manager.cc:1428] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260812 03:13:11.835553 20369 raft_consensus.cc:348] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } }
I20260812 03:13:11.835660 20369 raft_consensus.cc:374] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:11.835696 20369 raft_consensus.cc:729] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fce8ee963fb49419a82de19f9cdf1d0, State: Initialized, Role: FOLLOWER
I20260812 03:13:11.835813 20369 consensus_queue.cc:260] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } }
I20260812 03:13:11.836050 20369 ts_tablet_manager.cc:1428] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:13:11.836733 20377 tablet_replica.cc:439] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } }
I20260812 03:13:11.836885 20377 tablet_replica.cc:442] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:11.837261 20368 tablet_bootstrap.cc:492] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654: No bootstrap required, opened a new log
I20260812 03:13:11.837350 20368 ts_tablet_manager.cc:1397] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20260812 03:13:11.837764 20368 raft_consensus.cc:348] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } }
I20260812 03:13:11.837920 20368 raft_consensus.cc:374] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:11.837956 20368 raft_consensus.cc:729] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ac5d24cb3c54782b2d1ad87eb6a5654, State: Initialized, Role: FOLLOWER
I20260812 03:13:11.838076 20368 consensus_queue.cc:260] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } }
I20260812 03:13:11.838362 20368 ts_tablet_manager.cc:1428] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:11.838567 20376 tablet_replica.cc:439] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } }
I20260812 03:13:11.838716 20376 tablet_replica.cc:442] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:11.865320 20391 raft_consensus.cc:482] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:11.865459 20391 raft_consensus.cc:504] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } }
I20260812 03:13:11.865831 20391 leader_election.cc:288] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ac5d24cb3c54782b2d1ad87eb6a5654 (127.16.217.65:40839), 2fce8ee963fb49419a82de19f9cdf1d0 (127.16.217.66:45623)
I20260812 03:13:11.866639 19679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6efef2eb26b248028163ad874a0cca31" candidate_uuid: "357d8f112d114394bde95ab401f87c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" is_pre_election: true
I20260812 03:13:11.866639 19927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6efef2eb26b248028163ad874a0cca31" candidate_uuid: "357d8f112d114394bde95ab401f87c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" is_pre_election: true
I20260812 03:13:11.866804 19927 raft_consensus.cc:2454] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 357d8f112d114394bde95ab401f87c84 in term 0.
I20260812 03:13:11.866804 19679 raft_consensus.cc:2454] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 357d8f112d114394bde95ab401f87c84 in term 0.
I20260812 03:13:11.867170 20081 leader_election.cc:302] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 357d8f112d114394bde95ab401f87c84, 4ac5d24cb3c54782b2d1ad87eb6a5654; no voters:
I20260812 03:13:11.867444 20391 raft_consensus.cc:2781] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:13:11.867542 20391 raft_consensus.cc:482] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:13:11.867595 20391 raft_consensus.cc:3037] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:11.868728 20391 raft_consensus.cc:504] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } }
I20260812 03:13:11.869081 20391 leader_election.cc:288] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [CANDIDATE]: Term 1 election: Requested vote from peers 4ac5d24cb3c54782b2d1ad87eb6a5654 (127.16.217.65:40839), 2fce8ee963fb49419a82de19f9cdf1d0 (127.16.217.66:45623)
I20260812 03:13:11.869902 19679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6efef2eb26b248028163ad874a0cca31" candidate_uuid: "357d8f112d114394bde95ab401f87c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654"
I20260812 03:13:11.870050 19679 raft_consensus.cc:3037] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:11.870011 19927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6efef2eb26b248028163ad874a0cca31" candidate_uuid: "357d8f112d114394bde95ab401f87c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fce8ee963fb49419a82de19f9cdf1d0"
I20260812 03:13:11.870278 19927 raft_consensus.cc:3037] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:11.871428 19679 raft_consensus.cc:2454] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 357d8f112d114394bde95ab401f87c84 in term 1.
I20260812 03:13:11.871780 19927 raft_consensus.cc:2454] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 357d8f112d114394bde95ab401f87c84 in term 1.
I20260812 03:13:11.871827 20081 leader_election.cc:302] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 357d8f112d114394bde95ab401f87c84, 4ac5d24cb3c54782b2d1ad87eb6a5654; no voters:
I20260812 03:13:11.871994 20391 raft_consensus.cc:2781] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:11.872211 20391 raft_consensus.cc:686] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [term 1 LEADER]: Becoming Leader. State: Replica: 357d8f112d114394bde95ab401f87c84, State: Running, Role: LEADER
I20260812 03:13:11.872400 20391 consensus_queue.cc:237] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } }
I20260812 03:13:11.873701 20375 tablet_replica.cc:439] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84: TxnStatusTablet state changed. Reason: New leader 357d8f112d114394bde95ab401f87c84. Latest consensus state: current_term: 1 leader_uuid: "357d8f112d114394bde95ab401f87c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } }
I20260812 03:13:11.873823 20375 tablet_replica.cc:442] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:13:11.874696 20455 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 03:13:11.874904 20455 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 03:13:11.875491 19373 catalog_manager.cc:5495] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 357d8f112d114394bde95ab401f87c84 (127.16.217.67). New cstate: current_term: 1 leader_uuid: "357d8f112d114394bde95ab401f87c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } health_report { overall_health: HEALTHY } } }
I20260812 03:13:11.896178 20359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.892880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42298 (local address 127.16.217.67:35251)
0812 03:13:11.893017 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:11.893021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:11.893038 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:13:11.893098 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:11.893100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:11.893145 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:11.893232 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:11.893501 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.894035 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.894973 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.895207 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.895318 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.895609 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:11.895700 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:11.895783 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:11.895888 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 03:13:11.895940 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:13:11.896194 20441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.892645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:35251 (local address 127.0.0.1:42298)
0812 03:13:11.892804 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:11.892822 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:11.892934 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:11.893262 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:11.893265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:11.893272 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:13:11.893389 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.893394 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.894164 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.894167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.894848 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.894858 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.895322 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.895326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.895423 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.895469 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:13:11.895793 (+ 324us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:13:11.895811 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 03:13:11.895876 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 03:13:11.895968 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 03:13:11.899215 19679 raft_consensus.cc:1261] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 1 FOLLOWER]: Refusing update from remote peer 357d8f112d114394bde95ab401f87c84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:11.899205 19927 raft_consensus.cc:1261] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0 [term 1 FOLLOWER]: Refusing update from remote peer 357d8f112d114394bde95ab401f87c84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:11.899708 20391 consensus_queue.cc:1035] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:11.899873 20375 consensus_queue.cc:1035] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:11.901288 20377 tablet_replica.cc:439] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0: TxnStatusTablet state changed. Reason: New leader 357d8f112d114394bde95ab401f87c84. Latest consensus state: current_term: 1 leader_uuid: "357d8f112d114394bde95ab401f87c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } }
I20260812 03:13:11.901399 20377 tablet_replica.cc:442] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:11.902529 20376 tablet_replica.cc:439] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654: TxnStatusTablet state changed. Reason: New leader 357d8f112d114394bde95ab401f87c84. Latest consensus state: current_term: 1 leader_uuid: "357d8f112d114394bde95ab401f87c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } }
I20260812 03:13:11.902653 20376 tablet_replica.cc:442] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:11.904460 20391 tablet_replica.cc:439] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "357d8f112d114394bde95ab401f87c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } }
I20260812 03:13:11.904589 20391 tablet_replica.cc:442] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:13:11.906793 20375 tablet_replica.cc:439] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "357d8f112d114394bde95ab401f87c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } }
I20260812 03:13:11.906911 20375 tablet_replica.cc:442] T 6efef2eb26b248028163ad874a0cca31 P 357d8f112d114394bde95ab401f87c84: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:13:11.907119 20377 tablet_replica.cc:439] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "357d8f112d114394bde95ab401f87c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } }
I20260812 03:13:11.907251 20377 tablet_replica.cc:442] T 6efef2eb26b248028163ad874a0cca31 P 2fce8ee963fb49419a82de19f9cdf1d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:11.908049 20376 tablet_replica.cc:439] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "357d8f112d114394bde95ab401f87c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 } } peers { permanent_uuid: "2fce8ee963fb49419a82de19f9cdf1d0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45623 } } peers { permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 } } }
I20260812 03:13:11.908186 20376 tablet_replica.cc:442] T 6efef2eb26b248028163ad874a0cca31 P 4ac5d24cb3c54782b2d1ad87eb6a5654: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:11.917330 20362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.911537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44612 (local address 127.16.217.66:45623)
0812 03:13:11.911912 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:11.911918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:11.911934 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:13:11.912000 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:11.912002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:11.912050 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:11.912167 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:11.912442 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.913895 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.915245 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.915601 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.915743 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.916178 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:11.916802 (+ 624us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:11.916926 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:11.917114 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 03:13:11.917179 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295}
I20260812 03:13:11.929981 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.926034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:35251 (local address 127.16.217.66:39369)
0812 03:13:11.926387 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:11.926403 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:11.926521 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:11.926831 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:11.926833 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:11.926840 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:13:11.926935 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.926939 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.927997 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.928000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.928678 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.928688 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.929269 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.929272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.929342 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.929394 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:13:11.929707 (+ 313us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:13:11.929712 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:13:11.929764 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 03:13:11.929838 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":134,"threads_started":1}
I20260812 03:13:11.929991 20359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.926435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:39369 (local address 127.16.217.67:35251)
0812 03:13:11.926586 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:11.926591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:11.926608 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:13:11.926663 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:11.926666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:11.926722 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:11.926816 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:11.927049 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.927859 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.928803 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.929133 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.929238 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.929530 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:11.929618 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:11.929694 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:11.929795 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 03:13:11.929854 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 03:13:11.939471 19679 raft_consensus.cc:1261] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 1 FOLLOWER]: Refusing update from remote peer 2fce8ee963fb49419a82de19f9cdf1d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:11.939705 20223 raft_consensus.cc:1261] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84 [term 1 FOLLOWER]: Refusing update from remote peer 2fce8ee963fb49419a82de19f9cdf1d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:11.939994 20377 consensus_queue.cc:1035] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ac5d24cb3c54782b2d1ad87eb6a5654" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:11.940236 20424 consensus_queue.cc:1035] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "357d8f112d114394bde95ab401f87c84" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:11.968321 20362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.962959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:44993 (local address 127.16.217.66:45623)
0812 03:13:11.963114 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:11.963117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:11.963519 (+ 402us) server_negotiation.cc:407] Connection header received
0812 03:13:11.963648 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:11.963652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:11.963713 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:11.963800 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:11.964204 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.964873 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.966469 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.966684 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.966827 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.967364 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:11.967459 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:11.967559 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:11.967882 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 03:13:11.967943 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:13:11.968420 20477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:11.962876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:45623 (local address 127.16.217.67:44993)
0812 03:13:11.963421 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:11.963436 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:11.963547 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:11.963959 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:11.963962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:11.963970 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:13:11.964082 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.964088 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.964994 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.964997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.966019 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:11.966030 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:11.966837 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:11.966841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:11.966934 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:11.966993 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:13:11.967725 (+ 732us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:13:11.967755 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 03:13:11.967857 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 03:13:11.967944 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":119,"threads_started":1}
I20260812 03:13:11.997730 19372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:11.997926 19372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:12.000691 19372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d92e9b7db8b543f9a233403c3147b853: Sending DeleteTablet for 3 replicas of tablet bd6f8fd319664616a90f5e92034c0b6e
I20260812 03:13:12.002666 20179 tablet_service.cc:1514] Processing DeleteTablet for tablet bd6f8fd319664616a90f5e92034c0b6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:11 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:42288
I20260812 03:13:12.003651 20400 tablet_replica.cc:330] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84: stopping tablet replica
I20260812 03:13:12.003783 20400 raft_consensus.cc:2229] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:12.003899 20400 raft_consensus.cc:2258] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:12.004247 19905 tablet_service.cc:1514] Processing DeleteTablet for tablet bd6f8fd319664616a90f5e92034c0b6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:11 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:44600
I20260812 03:13:12.004436 20404 tablet_replica.cc:330] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0: stopping tablet replica
I20260812 03:13:12.004535 20404 raft_consensus.cc:2229] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:13:12.004554 19636 tablet_service.cc:1514] Processing DeleteTablet for tablet bd6f8fd319664616a90f5e92034c0b6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:11 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:40640
I20260812 03:13:12.004776 20408 tablet_replica.cc:330] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654: stopping tablet replica
I20260812 03:13:12.004894 20408 raft_consensus.cc:2229] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:12.004904 20404 raft_consensus.cc:2258] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:12.005014 20408 raft_consensus.cc:2258] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:12.005303 20400 ts_tablet_manager.cc:1905] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:12.005697 20404 ts_tablet_manager.cc:1905] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:12.005790 20408 ts_tablet_manager.cc:1905] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:12.008602 20404 ts_tablet_manager.cc:1918] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:13:12.008685 20404 log.cc:1196] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/bd6f8fd319664616a90f5e92034c0b6e
I20260812 03:13:12.008881 20408 ts_tablet_manager.cc:1918] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:13:12.008946 20404 ts_tablet_manager.cc:1939] T bd6f8fd319664616a90f5e92034c0b6e P 2fce8ee963fb49419a82de19f9cdf1d0: Deleting consensus metadata
I20260812 03:13:12.008966 20408 log.cc:1196] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-0/wal/wals/bd6f8fd319664616a90f5e92034c0b6e
I20260812 03:13:12.009233 20408 ts_tablet_manager.cc:1939] T bd6f8fd319664616a90f5e92034c0b6e P 4ac5d24cb3c54782b2d1ad87eb6a5654: Deleting consensus metadata
I20260812 03:13:12.009753 19354 catalog_manager.cc:4841] TS 2fce8ee963fb49419a82de19f9cdf1d0 (127.16.217.66:45623): tablet bd6f8fd319664616a90f5e92034c0b6e (table test-table [id=917bf53edb4d4211bddf062fc4dad3df]) successfully deleted
I20260812 03:13:12.009955 19354 catalog_manager.cc:4841] TS 4ac5d24cb3c54782b2d1ad87eb6a5654 (127.16.217.65:40839): tablet bd6f8fd319664616a90f5e92034c0b6e (table test-table [id=917bf53edb4d4211bddf062fc4dad3df]) successfully deleted
I20260812 03:13:12.010773 20400 ts_tablet_manager.cc:1918] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:13:12.010902 20400 log.cc:1196] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/bd6f8fd319664616a90f5e92034c0b6e
I20260812 03:13:12.011229 20400 ts_tablet_manager.cc:1939] T bd6f8fd319664616a90f5e92034c0b6e P 357d8f112d114394bde95ab401f87c84: Deleting consensus metadata
I20260812 03:13:12.012020 19348 catalog_manager.cc:4841] TS 357d8f112d114394bde95ab401f87c84 (127.16.217.67:35251): tablet bd6f8fd319664616a90f5e92034c0b6e (table test-table [id=917bf53edb4d4211bddf062fc4dad3df]) successfully deleted
Aug 12 03:13:12 dist-test-slave-7nm7 krb5kdc[19290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504391, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.217.65@KRBTEST.COM
I20260812 03:13:12.025478 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:12.012250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40652 (local address 127.16.217.65:40839)
0812 03:13:12.012375 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:12.012380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:12.012927 (+ 547us) server_negotiation.cc:407] Connection header received
0812 03:13:12.013135 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:12.013139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:12.013195 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:12.013267 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:12.014275 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:12.015058 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:12.016335 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:12.016676 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:12.016748 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:12.020868 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:12.020884 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:12.020907 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:12.020938 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:12.023121 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:12.023644 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:12.023651 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:12.023653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:12.023749 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:12.024107 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:12.024111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:12.024112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:12.024569 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:12.024827 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:12.025159 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 03:13:12.025294 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 03:13:12.026049 19743 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40652
I20260812 03:13:12.040603 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:12.030717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34610 (local address 127.16.217.126:40971)
0812 03:13:12.030860 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:12.030864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:12.031016 (+ 152us) server_negotiation.cc:407] Connection header received
0812 03:13:12.031145 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:12.031148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:12.031202 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:12.031283 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:12.032172 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:12.032966 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:12.034107 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:12.034485 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:12.034608 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:12.036222 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:12.036226 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:12.036229 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:12.036257 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:12.038583 (+ 2326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:12.039190 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:12.039194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:12.039196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:12.039291 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:12.039624 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:12.039628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:12.039630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:12.039850 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:12.039983 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:12.040282 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 03:13:12.040416 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 03:13:12.041035 19372 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34610
I20260812 03:13:12.042491 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 19513
I20260812 03:13:12.063802 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 19777
I20260812 03:13:12.081521 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 20045
I20260812 03:13:12.104643 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 19311
2026-08-12T03:13:12Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3482 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:13:12 dist-test-slave-7nm7 krb5kdc[20496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:13:12 dist-test-slave-7nm7 krb5kdc[20496](info): set up 2 sockets
Aug 12 03:13:12 dist-test-slave-7nm7 krb5kdc[20496](info): commencing operation
krb5kdc: starting...
W20260812 03:13:13.440116 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:13:13 dist-test-slave-7nm7 krb5kdc[20496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:13:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:13:13.576848 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:43383
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:36617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:43383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:13.696337 20526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:13.696637 20526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:13.696734 20526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:13.700793 20526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:13.700874 20526 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 03:13:13.700907 20526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:13.700965 20526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:13.701002 20526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:13.701040 20526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:13:13.706521 20526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:36617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:43383
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:43383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:13.707672 20526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:13.708724 20526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:13.717619 20538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:13.717574 20543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:13.717621 20541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:13.717792 20526 server_base.cc:1034] running on GCE node
I20260812 03:13:13.718633 20526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:13.719851 20526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:13.721021 20526 hybrid_clock.cc:648] HybridClock initialized: now 1786504393720998 us; error 46 us; skew 500 ppm
Aug 12 03:13:13 dist-test-slave-7nm7 krb5kdc[20496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504393, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:13.727334 20526 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:13:13.728364 20526 webserver.cc:460] Webserver started at http://127.16.217.126:37275/ using document root <none> and password file <none>
I20260812 03:13:13.728976 20526 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:13.729074 20526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:13.729329 20526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:13.731532 20526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "ca598746d4ce4dc19f8d928aefa19acc"
format_stamp: "Formatted at 2026-08-12 03:13:13 on dist-test-slave-7nm7"
server_key: "c83e84577edd3b3660f3d87463740f52"
server_key_iv: "4800c21e03fd6a770011c5b3f42bb975"
server_key_version: "encryptionkey@0"
I20260812 03:13:13.732147 20526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "ca598746d4ce4dc19f8d928aefa19acc"
format_stamp: "Formatted at 2026-08-12 03:13:13 on dist-test-slave-7nm7"
server_key: "c83e84577edd3b3660f3d87463740f52"
server_key_iv: "4800c21e03fd6a770011c5b3f42bb975"
server_key_version: "encryptionkey@0"
I20260812 03:13:13.736148 20526 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:13:13.738901 20556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:13.740281 20526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:13.740445 20526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "ca598746d4ce4dc19f8d928aefa19acc"
format_stamp: "Formatted at 2026-08-12 03:13:13 on dist-test-slave-7nm7"
server_key: "c83e84577edd3b3660f3d87463740f52"
server_key_iv: "4800c21e03fd6a770011c5b3f42bb975"
server_key_version: "encryptionkey@0"
I20260812 03:13:13.740572 20526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:13.764379 20526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:13.768222 20526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:13.768406 20526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:13.777244 20526 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:43383
I20260812 03:13:13.777341 20704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:43383 every 8 connection(s)
I20260812 03:13:13.778296 20526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:13.781304 20705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:13.786183 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 20526
I20260812 03:13:13.786338 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:13:13.786651 17253 external_mini_cluster.cc:1428] Setting key e214ae7d54f7111c4ad9f25e495e2578
I20260812 03:13:13.787554 20705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc: Bootstrap starting.
I20260812 03:13:13.790086 20705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:13.790889 20705 log.cc:824] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:13.793867 20705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc: No bootstrap required, opened a new log
Aug 12 03:13:13 dist-test-slave-7nm7 krb5kdc[20496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:13.798731 20705 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca598746d4ce4dc19f8d928aefa19acc" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43383 } }
I20260812 03:13:13.798972 20705 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:13.799043 20705 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca598746d4ce4dc19f8d928aefa19acc, State: Initialized, Role: FOLLOWER
I20260812 03:13:13.799614 20705 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca598746d4ce4dc19f8d928aefa19acc" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43383 } }
I20260812 03:13:13.799775 20705 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:13.799891 20705 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:13.800012 20705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:13.800979 20705 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca598746d4ce4dc19f8d928aefa19acc" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43383 } }
I20260812 03:13:13.801362 20705 leader_election.cc:302] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca598746d4ce4dc19f8d928aefa19acc; no voters:
I20260812 03:13:13.801748 20705 leader_election.cc:288] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:13:13.801818 20712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:13.802063 20712 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc [term 1 LEADER]: Becoming Leader. State: Replica: ca598746d4ce4dc19f8d928aefa19acc, State: Running, Role: LEADER
I20260812 03:13:13.802424 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:13.788225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52968 (local address 127.16.217.126:43383)
0812 03:13:13.788707 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:13.788723 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:13.788759 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:13:13.789421 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:13.789438 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:13.789798 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:13.790150 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:13.791144 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:13.792576 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:13.793499 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:13.794176 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:13.794415 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:13.797173 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:13.797178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:13.797192 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:13.797220 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:13.799116 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:13.799615 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:13.799619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:13.799624 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:13.799761 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:13.800119 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:13.800123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:13.800125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:13.800537 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:13.800764 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:13.801028 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 03:13:13.801385 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":321,"thread_start_us":156,"threads_started":1}
I20260812 03:13:13.802474 20712 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca598746d4ce4dc19f8d928aefa19acc" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43383 } }
I20260812 03:13:13.803162 20705 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:13.804180 20715 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca598746d4ce4dc19f8d928aefa19acc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca598746d4ce4dc19f8d928aefa19acc" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43383 } } }
I20260812 03:13:13.804335 20715 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:13.804795 20718 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca598746d4ce4dc19f8d928aefa19acc. Latest consensus state: current_term: 1 leader_uuid: "ca598746d4ce4dc19f8d928aefa19acc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca598746d4ce4dc19f8d928aefa19acc" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43383 } } }
I20260812 03:13:13.805416 20718 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:13.805760 20730 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:13.808852 20730 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:13.814870 20730 catalog_manager.cc:1347] Generated new cluster ID: 8626e5a663bd4392ab299355712d616e
I20260812 03:13:13.814982 20730 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:13.829856 20730 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:13:13.831200 20730 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:13.839044 20730 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc: Generated new TSK 0
I20260812 03:13:13.839784 20730 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
I20260812 03:13:13.874477 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:43383
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:36617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:14.015833 20751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:14.016096 20751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:14.016153 20751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:14.020210 20751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:14.020303 20751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:14.020380 20751 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 03:13:14.020423 20751 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:13:14.020494 20751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:13:14.025914 20751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:36617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.217.126:43383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:14.027015 20751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:14.028065 20751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:14.038369 20766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:14.038369 20769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:14.038467 20765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:14.038647 20751 server_base.cc:1034] running on GCE node
I20260812 03:13:14.039333 20751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:14.039960 20751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:14.041182 20751 hybrid_clock.cc:648] HybridClock initialized: now 1786504394041138 us; error 45 us; skew 500 ppm
Aug 12 03:13:14 dist-test-slave-7nm7 krb5kdc[20496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504394, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:14.045679 20751 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:13:14.046725 20751 webserver.cc:460] Webserver started at http://127.16.217.65:46855/ using document root <none> and password file <none>
I20260812 03:13:14.047341 20751 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:14.047438 20751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:14.047685 20751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:14.049782 20751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb"
format_stamp: "Formatted at 2026-08-12 03:13:14 on dist-test-slave-7nm7"
server_key: "c7a5a4bf01f396da8472073a4ed06d40"
server_key_iv: "b2c30ec61815dc98ff266be3064d2c70"
server_key_version: "encryptionkey@0"
I20260812 03:13:14.050426 20751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb"
format_stamp: "Formatted at 2026-08-12 03:13:14 on dist-test-slave-7nm7"
server_key: "c7a5a4bf01f396da8472073a4ed06d40"
server_key_iv: "b2c30ec61815dc98ff266be3064d2c70"
server_key_version: "encryptionkey@0"
I20260812 03:13:14.054399 20751 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:13:14.056811 20783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:14.058514 20751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:14.058674 20751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb"
format_stamp: "Formatted at 2026-08-12 03:13:14 on dist-test-slave-7nm7"
server_key: "c7a5a4bf01f396da8472073a4ed06d40"
server_key_iv: "b2c30ec61815dc98ff266be3064d2c70"
server_key_version: "encryptionkey@0"
I20260812 03:13:14.058796 20751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:14.083779 20751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:14.087971 20751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:14.088234 20751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:14.089844 20751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:14.089923 20751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:14.090000 20751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:14.090045 20751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:13:14 dist-test-slave-7nm7 krb5kdc[20496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504394, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:14.103135 20751 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:37809
I20260812 03:13:14.103238 21008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:37809 every 8 connection(s)
I20260812 03:13:14.105020 20751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:13:14.108471 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.090689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:46023 (local address 127.16.217.126:43383)
0812 03:13:14.090864 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:14.090869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:14.092510 (+ 1641us) server_negotiation.cc:407] Connection header received
0812 03:13:14.093770 (+ 1260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:14.093774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:14.093838 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:14.093932 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:14.095721 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.096332 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.098406 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.098668 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.098776 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.102885 (+ 4109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:14.102889 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:14.102892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:14.102932 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:14.105058 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:14.106178 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:14.106183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:14.106185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:14.106371 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:14.106989 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:14.106994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:14.106996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:14.107457 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:14.107603 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:14.108129 (+ 526us) server_negotiation.cc:299] Negotiation successful
0812 03:13:14.108290 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 03:13:14.109269 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.091717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43383 (local address 127.16.217.65:46023)
0812 03:13:14.092330 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:14.092383 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:14.093543 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:14.094166 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:14.094175 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:14.094735 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:14.095513 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.095538 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.096485 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.096493 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.098266 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.098283 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.098859 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.098866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.099101 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.099874 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:14.099900 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:14.102676 (+ 2776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:14.105432 (+ 2756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:14.105440 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:14.105463 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:14.105978 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:14.106615 (+ 637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:14.106620 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:14.106623 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:14.106844 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:14.107601 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:14.107608 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:14.107877 (+ 269us) client_negotiation.cc:769] Sending connection context
0812 03:13:14.108072 (+ 195us) client_negotiation.cc:240] Negotiation successful
0812 03:13:14.108435 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":146,"threads_started":1}
I20260812 03:13:14.110852 21010 heartbeater.cc:347] Connected to a master server at 127.16.217.126:43383
I20260812 03:13:14.111034 21010 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:14.111588 21010 heartbeater.cc:511] Master 127.16.217.126:43383 requested a full tablet report, sending...
I20260812 03:13:14.113206 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 20751
I20260812 03:13:14.113209 20615 ts_manager.cc:194] Registered new tserver with Master: 13a38e66ae0c4c1ebd5f572e22bbb7fb (127.16.217.65:37809)
I20260812 03:13:14.113400 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:13:14.114022 17253 external_mini_cluster.cc:1428] Setting key ed8f8e952bd9bcf0ae582d1064fa476a
I20260812 03:13:14.114825 20615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:46023
I20260812 03:13:14.124639 21020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.113462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43383 (local address 127.16.217.65:33219)
0812 03:13:14.113896 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:14.113913 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:14.114011 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:14.114609 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:14.114615 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:14.115051 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:14.115300 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.115311 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.116725 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.116729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.117450 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.117466 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.118182 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.118186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.118481 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.119209 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:14.119222 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:14.119642 (+ 420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:14.122482 (+ 2840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:14.122488 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:14.122491 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:14.122760 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:14.123267 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:14.123270 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:14.123271 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:14.123360 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:14.124140 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:14.124142 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:14.124239 (+ 97us) client_negotiation.cc:769] Sending connection context
0812 03:13:14.124335 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 03:13:14.124466 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":177,"threads_started":1}
I20260812 03:13:14.125021 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.114026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:33219 (local address 127.16.217.126:43383)
0812 03:13:14.114372 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:14.114376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:14.114389 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:13:14.114423 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:14.114426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:14.114466 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:14.114543 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:14.115523 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.116459 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.117648 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.118025 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.118176 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.119846 (+ 1670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:14.119851 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:14.119854 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:14.119891 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:14.122163 (+ 2272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:14.122904 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:14.122910 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:14.122913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:14.123017 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:14.123484 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:14.123491 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:14.123496 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:14.123976 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:14.124149 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:14.124572 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 03:13:14.124781 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":248}
WARNING: no policy specified for kudu/127.16.217.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
I20260812 03:13:14.150658 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:43383
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:36617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:14.278332 21024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:14.278623 21024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:14.278714 21024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:14.283334 21024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:14.283418 21024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:14.283465 21024 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 03:13:14.283480 21024 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:13:14.283565 21024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:13:14.290798 21024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:36617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.217.126:43383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:14.292039 21024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:14.293222 21024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:14.303966 21045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:14.303982 21037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:14.304196 21024 server_base.cc:1034] running on GCE node
W20260812 03:13:14.304028 21043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:14.304634 21024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:14.305424 21024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:14.306622 21024 hybrid_clock.cc:648] HybridClock initialized: now 1786504394306598 us; error 45 us; skew 500 ppm
Aug 12 03:13:14 dist-test-slave-7nm7 krb5kdc[20496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504394, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:14.309530 21024 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:13:14.310576 21024 webserver.cc:460] Webserver started at http://127.16.217.66:45101/ using document root <none> and password file <none>
I20260812 03:13:14.311182 21024 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:14.311290 21024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:14.311524 21024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:14.313596 21024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "b9c0ab645cc94a4eb4620fc74d17004e"
format_stamp: "Formatted at 2026-08-12 03:13:14 on dist-test-slave-7nm7"
server_key: "f43344f4016d44de78ee873170e1ebbb"
server_key_iv: "f13dcc970413f243f8d1906ca0c24652"
server_key_version: "encryptionkey@0"
I20260812 03:13:14.314123 21024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "b9c0ab645cc94a4eb4620fc74d17004e"
format_stamp: "Formatted at 2026-08-12 03:13:14 on dist-test-slave-7nm7"
server_key: "f43344f4016d44de78ee873170e1ebbb"
server_key_iv: "f13dcc970413f243f8d1906ca0c24652"
server_key_version: "encryptionkey@0"
I20260812 03:13:14.318360 21024 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:13:14.320883 21055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:14.322516 21024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:14.322633 21024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "b9c0ab645cc94a4eb4620fc74d17004e"
format_stamp: "Formatted at 2026-08-12 03:13:14 on dist-test-slave-7nm7"
server_key: "f43344f4016d44de78ee873170e1ebbb"
server_key_iv: "f13dcc970413f243f8d1906ca0c24652"
server_key_version: "encryptionkey@0"
I20260812 03:13:14.322731 21024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:14.354077 21024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:14.357580 21024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:14.357749 21024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:14.359333 21024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:14.359387 21024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:14.359429 21024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:14.359481 21024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:13:14 dist-test-slave-7nm7 krb5kdc[20496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504394, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:14.370982 21024 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:33493
I20260812 03:13:14.371047 21292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:33493 every 8 connection(s)
I20260812 03:13:14.372419 21024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:13:14.375146 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.360092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:34179 (local address 127.16.217.126:43383)
0812 03:13:14.360249 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:14.360255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:14.361328 (+ 1073us) server_negotiation.cc:407] Connection header received
0812 03:13:14.363001 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:14.363007 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:14.363075 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:14.363175 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:14.364864 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.365694 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.367063 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.367308 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.367434 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.370698 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:14.370705 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:14.370709 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:14.370744 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:14.372720 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:14.373399 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:14.373402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:14.373404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:14.373496 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:14.373896 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:14.373898 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:14.373899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:14.374077 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:14.374153 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:14.374922 (+ 769us) server_negotiation.cc:299] Negotiation successful
0812 03:13:14.375023 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:13:14.375978 21081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.360637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43383 (local address 127.16.217.66:34179)
0812 03:13:14.361163 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:14.361205 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:14.362611 (+ 1406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:14.363437 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:14.363446 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:14.363857 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:14.364687 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.364724 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.365812 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.365817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.366941 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.366951 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.367499 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.367503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.367709 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.368425 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:14.368443 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:14.370476 (+ 2033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:14.372867 (+ 2391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:14.372875 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:14.372891 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:14.373270 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:14.373621 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:14.373624 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:14.373625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:14.373785 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:14.374190 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:14.374195 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:14.374707 (+ 512us) client_negotiation.cc:769] Sending connection context
0812 03:13:14.374859 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 03:13:14.375126 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":146,"threads_started":1}
I20260812 03:13:14.377292 21293 heartbeater.cc:347] Connected to a master server at 127.16.217.126:43383
I20260812 03:13:14.377477 21293 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:14.377978 21293 heartbeater.cc:511] Master 127.16.217.126:43383 requested a full tablet report, sending...
I20260812 03:13:14.379101 20615 ts_manager.cc:194] Registered new tserver with Master: b9c0ab645cc94a4eb4620fc74d17004e (127.16.217.66:33493)
I20260812 03:13:14.379848 20615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:34179
I20260812 03:13:14.380735 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 21024
I20260812 03:13:14.380847 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:13:14.381155 17253 external_mini_cluster.cc:1428] Setting key de196ede2b476ef452c4ad1b5acbc191
I20260812 03:13:14.387799 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.379582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:37345 (local address 127.16.217.126:43383)
0812 03:13:14.379700 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:14.379703 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:14.379921 (+ 218us) server_negotiation.cc:407] Connection header received
0812 03:13:14.380062 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:14.380065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:14.380131 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:14.380217 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:14.380751 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.381449 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.382558 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.382781 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.382862 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.384227 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:14.384231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:14.384232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:14.384267 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:14.386085 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:14.386560 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:14.386563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:14.386565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:14.386651 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:14.386936 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:14.386940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:14.386942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:14.387161 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:14.387277 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:14.387538 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 03:13:14.387647 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:13:14.387811 21307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.379500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43383 (local address 127.16.217.66:37345)
0812 03:13:14.379832 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:14.379843 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:14.379969 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:14.380235 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:14.380238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:14.380478 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:14.380617 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.380625 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.381605 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.381608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.382438 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.382448 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.382909 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.382913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.383052 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.383604 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:14.383624 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:14.384088 (+ 464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:14.386187 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:14.386190 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:14.386192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:14.386450 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:14.386753 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:14.386757 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:14.386759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:14.386843 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:14.387269 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:14.387272 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:14.387371 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 03:13:14.387482 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 03:13:14.387630 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":101,"threads_started":1}
WARNING: no policy specified for kudu/127.16.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
I20260812 03:13:14.411278 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:43383
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:36617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:14.532739 21311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:14.533017 21311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:14.533066 21311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:14.537075 21311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:14.537137 21311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:14.537177 21311 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 03:13:14.537238 21311 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:13:14.537293 21311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:13:14.542558 21311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:36617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.217.126:43383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:14.543823 21311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:14.544991 21311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:14.555315 21317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:14.555333 21321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:14.555387 21311 server_base.cc:1034] running on GCE node
W20260812 03:13:14.555333 21318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:14.556010 21311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:14.556699 21311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:14.557844 21311 hybrid_clock.cc:648] HybridClock initialized: now 1786504394557812 us; error 42 us; skew 500 ppm
Aug 12 03:13:14 dist-test-slave-7nm7 krb5kdc[20496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504394, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:14.560580 21311 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:13:14.561451 21311 webserver.cc:460] Webserver started at http://127.16.217.67:39741/ using document root <none> and password file <none>
I20260812 03:13:14.561975 21311 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:14.562032 21311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:14.562311 21311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:14.564405 21311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "9089de23903d46ef92e1e327f664f42a"
format_stamp: "Formatted at 2026-08-12 03:13:14 on dist-test-slave-7nm7"
server_key: "f04518572dfe39258881b2221e1ce5ce"
server_key_iv: "f7c6526f1bcf65e6733d0a69d89c8d52"
server_key_version: "encryptionkey@0"
I20260812 03:13:14.564931 21311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "9089de23903d46ef92e1e327f664f42a"
format_stamp: "Formatted at 2026-08-12 03:13:14 on dist-test-slave-7nm7"
server_key: "f04518572dfe39258881b2221e1ce5ce"
server_key_iv: "f7c6526f1bcf65e6733d0a69d89c8d52"
server_key_version: "encryptionkey@0"
I20260812 03:13:14.569013 21311 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 03:13:14.571489 21334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:14.572724 21311 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:13:14.572853 21311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "9089de23903d46ef92e1e327f664f42a"
format_stamp: "Formatted at 2026-08-12 03:13:14 on dist-test-slave-7nm7"
server_key: "f04518572dfe39258881b2221e1ce5ce"
server_key_iv: "f7c6526f1bcf65e6733d0a69d89c8d52"
server_key_version: "encryptionkey@0"
I20260812 03:13:14.572963 21311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:14.589782 21311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:14.593150 21311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:14.593364 21311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:14.595065 21311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:14.595148 21311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:14.595224 21311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:14.595264 21311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:13:14 dist-test-slave-7nm7 krb5kdc[20496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504394, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:14.607530 21311 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:44107
I20260812 03:13:14.607758 21556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:44107 every 8 connection(s)
I20260812 03:13:14.608857 21311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:13:14.611233 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.595756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:34017 (local address 127.16.217.126:43383)
0812 03:13:14.595932 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:14.595937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:14.597444 (+ 1507us) server_negotiation.cc:407] Connection header received
0812 03:13:14.598987 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:14.598991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:14.599060 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:14.599200 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:14.600963 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.601567 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.603104 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.603334 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.603452 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.606962 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:14.606966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:14.606968 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:14.606999 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:14.608882 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:14.609480 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:14.609483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:14.609485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:14.609594 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:14.610027 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:14.610030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:14.610032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:14.610353 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:14.610440 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:14.610961 (+ 521us) server_negotiation.cc:299] Negotiation successful
0812 03:13:14.611093 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:13:14.612284 21357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.596675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43383 (local address 127.16.217.67:34017)
0812 03:13:14.597278 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:14.597337 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:14.598712 (+ 1375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:14.599386 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:14.599397 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:14.599939 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:14.600743 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.600768 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.601720 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.601724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.602975 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.602985 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.603501 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.603505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.603696 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.604659 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:14.604690 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:14.606777 (+ 2087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:14.609025 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:14.609030 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:14.609043 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:14.609355 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:14.609709 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:14.609712 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:14.609714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:14.609896 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:14.610454 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:14.610460 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:14.610698 (+ 238us) client_negotiation.cc:769] Sending connection context
0812 03:13:14.610864 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 03:13:14.611248 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":137,"threads_started":1}
I20260812 03:13:14.613627 21557 heartbeater.cc:347] Connected to a master server at 127.16.217.126:43383
I20260812 03:13:14.613859 21557 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:14.614497 21557 heartbeater.cc:511] Master 127.16.217.126:43383 requested a full tablet report, sending...
I20260812 03:13:14.615684 20613 ts_manager.cc:194] Registered new tserver with Master: 9089de23903d46ef92e1e327f664f42a (127.16.217.67:44107)
I20260812 03:13:14.616374 20613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:34017
I20260812 03:13:14.618952 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 21311
I20260812 03:13:14.619114 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:13:14.619436 17253 external_mini_cluster.cc:1428] Setting key da6f327d07d4130fa2ab98083436cfe4
I20260812 03:13:14.622143 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:13:14.626015 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.616337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:36469 (local address 127.16.217.126:43383)
0812 03:13:14.616472 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:14.616478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:14.616704 (+ 226us) server_negotiation.cc:407] Connection header received
0812 03:13:14.616796 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:14.616799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:14.616846 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:14.616936 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:14.617428 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.618465 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.619693 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.620017 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.620144 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.621871 (+ 1727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:14.621876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:14.621878 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:14.621915 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:14.624083 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:14.624611 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:14.624617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:14.624622 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:14.624736 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:14.625012 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:14.625017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:14.625020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:14.625277 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:14.625373 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:14.625753 (+ 380us) server_negotiation.cc:299] Negotiation successful
0812 03:13:14.625850 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:13:14.626148 21571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.616253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43383 (local address 127.16.217.67:36469)
0812 03:13:14.616614 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:14.616626 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:14.616712 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:14.616966 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:14.616968 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:14.617186 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:14.617284 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.617290 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.618630 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.618633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.619547 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.619560 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.620158 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.620162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.620297 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.621076 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:14.621107 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:14.621601 (+ 494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:14.624233 (+ 2632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:14.624236 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:14.624238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:14.624491 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:14.624835 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:14.624837 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:14.624839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:14.624918 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:14.625410 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:14.625416 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:14.625536 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 03:13:14.625681 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 03:13:14.625867 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":103,"threads_started":1}
Aug 12 03:13:14 dist-test-slave-7nm7 krb5kdc[20496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:13:14 dist-test-slave-7nm7 krb5kdc[20496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:14.651928 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.642648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54106 (local address 127.16.217.126:43383)
0812 03:13:14.642784 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:14.642787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:14.642799 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:13:14.642841 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:14.642844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:14.642884 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:14.642980 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:14.643979 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.644563 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.645520 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.645735 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.645846 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.648720 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:14.648723 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:14.648725 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:14.648750 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:14.650373 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:14.650789 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:14.650791 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:14.650793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:14.650865 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:14.651165 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:14.651168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:14.651169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:14.651358 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:14.651469 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:14.651700 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 03:13:14.651805 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:13:14.655104 20613 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:13:14.658085 20613 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:13:14.673666 21602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.667820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51302 (local address 127.16.217.67:44107)
0812 03:13:14.668706 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:14.668719 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:14.668744 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:13:14.668823 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:14.668828 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:14.669038 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:14.669203 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:14.669463 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.670582 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.671902 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.672784 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.672913 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.673093 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:14.673383 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 03:13:14.673496 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":784,"thread_start_us":196,"threads_started":1}
I20260812 03:13:14.673669 21595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.667390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:44107 (local address 127.0.0.1:51302)
0812 03:13:14.668329 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:14.668346 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:14.668445 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:14.669203 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:14.669206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:14.669227 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:14.669330 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.669338 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.670728 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.670732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.671763 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.671774 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.672929 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.672933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.673155 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.673202 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 03:13:14.673329 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 03:13:14.673487 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"mutex_wait_us":106,"thread_start_us":98,"threads_started":1}
I20260812 03:13:14.673669 21594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.667098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:33493 (local address 127.0.0.1:58934)
0812 03:13:14.668159 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:14.668171 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:14.668309 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:14.668926 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:14.668929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:14.668942 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:14.669045 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.669054 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.670195 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.670202 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.671878 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.671891 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.672929 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.672933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.673113 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.673201 (+ 88us) client_negotiation.cc:769] Sending connection context
0812 03:13:14.673329 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 03:13:14.673483 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"mutex_wait_us":188,"thread_start_us":123,"threads_started":1}
I20260812 03:13:14.674657 21596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.667819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58934 (local address 127.16.217.66:33493)
0812 03:13:14.668310 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:14.668323 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:14.668384 (+ 61us) server_negotiation.cc:407] Connection header received
0812 03:13:14.668480 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:14.668489 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:14.668685 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:14.668847 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:14.669179 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.670066 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.672016 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.672790 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.672913 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.673094 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:14.674370 (+ 1276us) server_negotiation.cc:299] Negotiation successful
0812 03:13:14.674482 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":162,"threads_started":1}
I20260812 03:13:14.674787 21592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.666970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:37809 (local address 127.0.0.1:53900)
0812 03:13:14.667537 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:14.667600 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:14.667733 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:14.668783 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:14.668789 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:14.668811 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:14.668947 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.668955 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.670468 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.670472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.671883 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.671891 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.674342 (+ 2451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.674346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.674439 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.674465 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 03:13:14.674550 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 03:13:14.674617 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":141,"thread_start_us":115,"threads_started":1}
I20260812 03:13:14.675235 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.667088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53900 (local address 127.16.217.65:37809)
0812 03:13:14.667739 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:14.667746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:14.667771 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:13:14.667954 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:14.667960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:14.668602 (+ 642us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:14.668770 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:14.669090 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.670332 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.673171 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.673944 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.674080 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.674268 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:14.674985 (+ 717us) server_negotiation.cc:299] Negotiation successful
0812 03:13:14.675091 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":86,"threads_started":1}
I20260812 03:13:14.677230 21446 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1d1a308db3485296f5bc65275c6444 (DEFAULT_TABLE table=test-table [id=4b7b63fa0f9a4e4d89e6e941d45522fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:14.677347 21163 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1d1a308db3485296f5bc65275c6444 (DEFAULT_TABLE table=test-table [id=4b7b63fa0f9a4e4d89e6e941d45522fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:14.677765 20877 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1d1a308db3485296f5bc65275c6444 (DEFAULT_TABLE table=test-table [id=4b7b63fa0f9a4e4d89e6e941d45522fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:14.678458 21163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e1d1a308db3485296f5bc65275c6444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:14.678470 21446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e1d1a308db3485296f5bc65275c6444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:14.678723 20877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e1d1a308db3485296f5bc65275c6444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:14.685106 21604 tablet_bootstrap.cc:492] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a: Bootstrap starting.
I20260812 03:13:14.686518 21605 tablet_bootstrap.cc:492] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Bootstrap starting.
I20260812 03:13:14.686812 21607 tablet_bootstrap.cc:492] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e: Bootstrap starting.
I20260812 03:13:14.687813 21604 tablet_bootstrap.cc:654] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:14.688598 21605 tablet_bootstrap.cc:654] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:14.688727 21607 tablet_bootstrap.cc:654] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:14.689265 21604 log.cc:824] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:14.689383 21605 log.cc:824] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:14.689618 21607 log.cc:824] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:14.691440 21604 tablet_bootstrap.cc:492] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a: No bootstrap required, opened a new log
I20260812 03:13:14.691437 21605 tablet_bootstrap.cc:492] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: No bootstrap required, opened a new log
I20260812 03:13:14.691643 21604 ts_tablet_manager.cc:1397] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 03:13:14.691643 21605 ts_tablet_manager.cc:1397] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:13:14.691670 21607 tablet_bootstrap.cc:492] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e: No bootstrap required, opened a new log
I20260812 03:13:14.691879 21607 ts_tablet_manager.cc:1397] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 03:13:14.694878 21607 raft_consensus.cc:348] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:14.695138 21607 raft_consensus.cc:374] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:14.695207 21607 raft_consensus.cc:729] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9c0ab645cc94a4eb4620fc74d17004e, State: Initialized, Role: FOLLOWER
I20260812 03:13:14.695746 21607 consensus_queue.cc:260] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:14.695894 21604 raft_consensus.cc:348] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:14.696159 21604 raft_consensus.cc:374] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:14.696216 21604 raft_consensus.cc:729] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9089de23903d46ef92e1e327f664f42a, State: Initialized, Role: FOLLOWER
I20260812 03:13:14.696619 21604 consensus_queue.cc:260] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:14.696550 21605 raft_consensus.cc:348] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:14.696775 21605 raft_consensus.cc:374] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:14.696833 21605 raft_consensus.cc:729] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13a38e66ae0c4c1ebd5f572e22bbb7fb, State: Initialized, Role: FOLLOWER
I20260812 03:13:14.696893 21607 ts_tablet_manager.cc:1428] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 03:13:14.697278 21293 heartbeater.cc:503] Master 127.16.217.126:43383 was elected leader, sending a full tablet report...
I20260812 03:13:14.697255 21605 consensus_queue.cc:260] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:14.697558 21604 ts_tablet_manager.cc:1428] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260812 03:13:14.697903 21557 heartbeater.cc:503] Master 127.16.217.126:43383 was elected leader, sending a full tablet report...
I20260812 03:13:14.698149 21605 ts_tablet_manager.cc:1428] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 03:13:14.698518 21010 heartbeater.cc:503] Master 127.16.217.126:43383 was elected leader, sending a full tablet report...
I20260812 03:13:14.835273 21619 raft_consensus.cc:482] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:14.835551 21619 raft_consensus.cc:504] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:14.836769 21619 leader_election.cc:288] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13a38e66ae0c4c1ebd5f572e22bbb7fb (127.16.217.65:37809), b9c0ab645cc94a4eb4620fc74d17004e (127.16.217.66:33493)
I20260812 03:13:14.841393 21596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.837302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:43861 (local address 127.16.217.66:33493)
0812 03:13:14.837465 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:14.837469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:14.837492 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:13:14.837563 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:14.837566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:14.837614 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:14.837705 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:14.838030 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.838808 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.840158 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.840799 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.840942 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.841030 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:14.841221 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 03:13:14.841286 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 03:13:14.841432 21357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.837095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:33493 (local address 127.16.217.67:43861)
0812 03:13:14.837252 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:14.837269 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:14.837374 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:14.837718 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:14.837720 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:14.837738 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:14.837899 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.837908 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.838885 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.838888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.840041 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.840050 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.840913 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.840916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.841081 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.841112 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 03:13:14.841199 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 03:13:14.841293 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":19}
I20260812 03:13:14.842193 21621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.837095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:37809 (local address 127.16.217.67:56739)
0812 03:13:14.837644 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:14.837656 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:14.837801 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:14.838078 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:14.838080 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:14.838096 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:14.838720 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.838729 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.839727 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.839730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.840918 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.840929 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.841849 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.841854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.841942 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.841961 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 03:13:14.842023 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 03:13:14.842072 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":196,"thread_start_us":155,"threads_started":1}
I20260812 03:13:14.842303 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.837201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:56739 (local address 127.16.217.65:37809)
0812 03:13:14.837337 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:14.837342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:14.837758 (+ 416us) server_negotiation.cc:407] Connection header received
0812 03:13:14.837927 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:14.837933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:14.837997 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:14.838071 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:14.838837 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.839599 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.841087 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.841729 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.841861 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.841913 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:14.842064 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 03:13:14.842141 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:13:14.842206 21201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1d1a308db3485296f5bc65275c6444" candidate_uuid: "9089de23903d46ef92e1e327f664f42a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" is_pre_election: true
I20260812 03:13:14.842576 21201 raft_consensus.cc:2454] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9089de23903d46ef92e1e327f664f42a in term 0.
I20260812 03:13:14.842890 20911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1d1a308db3485296f5bc65275c6444" candidate_uuid: "9089de23903d46ef92e1e327f664f42a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" is_pre_election: true
I20260812 03:13:14.843159 20911 raft_consensus.cc:2454] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9089de23903d46ef92e1e327f664f42a in term 0.
I20260812 03:13:14.843160 21339 leader_election.cc:302] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9089de23903d46ef92e1e327f664f42a, b9c0ab645cc94a4eb4620fc74d17004e; no voters:
I20260812 03:13:14.843472 21619 raft_consensus.cc:2781] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:13:14.843585 21619 raft_consensus.cc:482] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:13:14.843665 21619 raft_consensus.cc:3037] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:14.845115 21619 raft_consensus.cc:504] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:14.845526 21619 leader_election.cc:288] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [CANDIDATE]: Term 1 election: Requested vote from peers 13a38e66ae0c4c1ebd5f572e22bbb7fb (127.16.217.65:37809), b9c0ab645cc94a4eb4620fc74d17004e (127.16.217.66:33493)
I20260812 03:13:14.845942 21201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1d1a308db3485296f5bc65275c6444" candidate_uuid: "9089de23903d46ef92e1e327f664f42a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c0ab645cc94a4eb4620fc74d17004e"
I20260812 03:13:14.846073 20911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1d1a308db3485296f5bc65275c6444" candidate_uuid: "9089de23903d46ef92e1e327f664f42a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb"
I20260812 03:13:14.846168 21201 raft_consensus.cc:3037] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:14.846210 20911 raft_consensus.cc:3037] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:14.847275 20911 raft_consensus.cc:2454] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9089de23903d46ef92e1e327f664f42a in term 1.
I20260812 03:13:14.847651 21340 leader_election.cc:302] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13a38e66ae0c4c1ebd5f572e22bbb7fb, 9089de23903d46ef92e1e327f664f42a; no voters:
I20260812 03:13:14.847711 21201 raft_consensus.cc:2454] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9089de23903d46ef92e1e327f664f42a in term 1.
I20260812 03:13:14.847890 21619 raft_consensus.cc:2781] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:14.848160 21619 raft_consensus.cc:686] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [term 1 LEADER]: Becoming Leader. State: Replica: 9089de23903d46ef92e1e327f664f42a, State: Running, Role: LEADER
I20260812 03:13:14.848544 21619 consensus_queue.cc:237] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:14.851967 20610 catalog_manager.cc:5495] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9089de23903d46ef92e1e327f664f42a (127.16.217.67). New cstate: current_term: 1 leader_uuid: "9089de23903d46ef92e1e327f664f42a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } health_report { overall_health: HEALTHY } } }
W20260812 03:13:14.857188 21011 tablet.cc:2369] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:13:14.860584 21560 tablet.cc:2369] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:13:14.874154 21296 tablet.cc:2369] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 03:13:14 dist-test-slave-7nm7 krb5kdc[20496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504393, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:14.940560 21639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.932133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43383 (local address 127.0.0.1:54114)
0812 03:13:14.932436 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:14.932449 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:14.932551 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:14.932796 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:14.932798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:14.933182 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:14.933281 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.933287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.934031 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.934034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.934704 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:14.934713 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.935175 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.935178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:14.935246 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.935798 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:14.935811 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:14.937256 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:14.938932 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:14.938945 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:14.938954 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:14.939215 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:14.939496 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:14.939498 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:14.939499 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:14.939581 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:14.940023 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:14.940029 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:14.940149 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 03:13:14.940253 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 03:13:14.940364 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":131,"threads_started":1}
I20260812 03:13:14.940569 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:14.932164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54114 (local address 127.16.217.126:43383)
0812 03:13:14.932320 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:14.932325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:14.932529 (+ 204us) server_negotiation.cc:407] Connection header received
0812 03:13:14.932648 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:14.932650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:14.932691 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:14.932761 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:14.933386 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.933908 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.934828 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:14.935034 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:14.935140 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:14.937382 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:14.937385 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:14.937387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:14.937408 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:14.938804 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:14.939316 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:14.939319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:14.939320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:14.939404 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:14.939665 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:14.939668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:14.939669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:14.939839 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:14.940018 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:14.940313 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 03:13:14.940420 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":54}
I20260812 03:13:14.945192 20610 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:54114:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 03:13:14.945912 20610 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:13:14.951063 21163 tablet_service.cc:1467] Processing CreateTablet for tablet 02a88b10313f4380b1b46f7b4e8641c6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a2ba40bb7c254078aab3b85b77a3d230]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:13:14.950950 20877 tablet_service.cc:1467] Processing CreateTablet for tablet 02a88b10313f4380b1b46f7b4e8641c6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a2ba40bb7c254078aab3b85b77a3d230]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:13:14.951229 21446 tablet_service.cc:1467] Processing CreateTablet for tablet 02a88b10313f4380b1b46f7b4e8641c6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a2ba40bb7c254078aab3b85b77a3d230]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:13:14.951742 21163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a88b10313f4380b1b46f7b4e8641c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:14.951742 20877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a88b10313f4380b1b46f7b4e8641c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:14.951875 21446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a88b10313f4380b1b46f7b4e8641c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:14.954345 21604 tablet_bootstrap.cc:492] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a: Bootstrap starting.
I20260812 03:13:14.954972 21605 tablet_bootstrap.cc:492] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Bootstrap starting.
I20260812 03:13:14.955466 21604 tablet_bootstrap.cc:654] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:14.955856 21607 tablet_bootstrap.cc:492] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e: Bootstrap starting.
I20260812 03:13:14.956030 21605 tablet_bootstrap.cc:654] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:14.956809 21607 tablet_bootstrap.cc:654] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:14.956948 21604 tablet_bootstrap.cc:492] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a: No bootstrap required, opened a new log
I20260812 03:13:14.957050 21604 ts_tablet_manager.cc:1397] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 03:13:14.957506 21605 tablet_bootstrap.cc:492] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: No bootstrap required, opened a new log
I20260812 03:13:14.957613 21605 ts_tablet_manager.cc:1397] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:14.957561 21604 raft_consensus.cc:348] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:14.957703 21604 raft_consensus.cc:374] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:14.957748 21604 raft_consensus.cc:729] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9089de23903d46ef92e1e327f664f42a, State: Initialized, Role: FOLLOWER
I20260812 03:13:14.957886 21604 consensus_queue.cc:260] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:14.958102 21605 raft_consensus.cc:348] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:14.958171 21604 ts_tablet_manager.cc:1428] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260812 03:13:14.958225 21605 raft_consensus.cc:374] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:14.958161 21619 tablet_replica.cc:439] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } } }
I20260812 03:13:14.958284 21605 raft_consensus.cc:729] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13a38e66ae0c4c1ebd5f572e22bbb7fb, State: Initialized, Role: FOLLOWER
I20260812 03:13:14.958320 21619 tablet_replica.cc:442] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:14.958422 21605 consensus_queue.cc:260] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:14.958675 21605 ts_tablet_manager.cc:1428] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:13:14.958693 21620 tablet_replica.cc:439] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } } }
I20260812 03:13:14.958840 21620 tablet_replica.cc:442] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:14.959486 21607 tablet_bootstrap.cc:492] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e: No bootstrap required, opened a new log
I20260812 03:13:14.959604 21607 ts_tablet_manager.cc:1397] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 03:13:14.960089 21607 raft_consensus.cc:348] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:14.960204 21607 raft_consensus.cc:374] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:14.960238 21607 raft_consensus.cc:729] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9c0ab645cc94a4eb4620fc74d17004e, State: Initialized, Role: FOLLOWER
I20260812 03:13:14.960384 21607 consensus_queue.cc:260] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:14.960813 21607 ts_tablet_manager.cc:1428] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:14.961117 21615 tablet_replica.cc:439] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } } }
I20260812 03:13:14.961270 21615 tablet_replica.cc:442] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:15.188479 21620 raft_consensus.cc:482] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:15.188695 21620 raft_consensus.cc:504] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:15.190301 21620 leader_election.cc:288] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9c0ab645cc94a4eb4620fc74d17004e (127.16.217.66:33493), 9089de23903d46ef92e1e327f664f42a (127.16.217.67:44107)
I20260812 03:13:15.195760 21596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:15.190545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:54811 (local address 127.16.217.66:33493)
0812 03:13:15.190752 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:15.190757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:15.190877 (+ 120us) server_negotiation.cc:407] Connection header received
0812 03:13:15.191153 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:15.191157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:15.191229 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:15.191328 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:15.191693 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.192569 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.194461 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.195063 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.195224 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:15.195283 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:15.195556 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 03:13:15.195629 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260812 03:13:15.195772 21647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:15.190759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:42941 (local address 127.16.217.67:44107)
0812 03:13:15.191018 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:15.191068 (+ 50us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:15.191158 (+ 90us) server_negotiation.cc:407] Connection header received
0812 03:13:15.191270 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:15.191277 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:15.191336 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:15.191428 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:15.191693 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.192581 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.194461 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.195102 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.195225 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:15.195281 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:15.195556 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 03:13:15.195639 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":76,"threads_started":1}
I20260812 03:13:15.195845 21644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:15.190412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:33493 (local address 127.16.217.65:54811)
0812 03:13:15.190786 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:15.190804 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:15.190977 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:15.191344 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:15.191346 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:15.191361 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:15.191568 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:15.191573 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.192708 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.192711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:15.194312 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:15.194324 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.195234 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.195238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:15.195431 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:15.195447 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 03:13:15.195522 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 03:13:15.195629 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":289,"thread_start_us":88,"threads_started":1}
I20260812 03:13:15.195853 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:15.190645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:44107 (local address 127.16.217.65:42941)
0812 03:13:15.191057 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:15.191071 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:15.191170 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:15.191441 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:15.191444 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:15.191456 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:15.191564 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:15.191572 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.192695 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.192700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:15.194312 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:15.194324 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.195234 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.195238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:15.195407 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:15.195440 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 03:13:15.195522 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:13:15.195629 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":215,"thread_start_us":90,"threads_started":1}
I20260812 03:13:15.196519 21201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a88b10313f4380b1b46f7b4e8641c6" candidate_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" is_pre_election: true
I20260812 03:13:15.196703 21201 raft_consensus.cc:2454] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13a38e66ae0c4c1ebd5f572e22bbb7fb in term 0.
I20260812 03:13:15.196905 21480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a88b10313f4380b1b46f7b4e8641c6" candidate_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9089de23903d46ef92e1e327f664f42a" is_pre_election: true
I20260812 03:13:15.197060 20786 leader_election.cc:302] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13a38e66ae0c4c1ebd5f572e22bbb7fb, b9c0ab645cc94a4eb4620fc74d17004e; no voters:
I20260812 03:13:15.197154 21480 raft_consensus.cc:2454] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13a38e66ae0c4c1ebd5f572e22bbb7fb in term 0.
I20260812 03:13:15.197363 21620 raft_consensus.cc:2781] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:13:15.197463 21620 raft_consensus.cc:482] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:13:15.197511 21620 raft_consensus.cc:3037] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:15.198549 21620 raft_consensus.cc:504] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:15.198843 21620 leader_election.cc:288] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [CANDIDATE]: Term 1 election: Requested vote from peers b9c0ab645cc94a4eb4620fc74d17004e (127.16.217.66:33493), 9089de23903d46ef92e1e327f664f42a (127.16.217.67:44107)
I20260812 03:13:15.199232 21480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a88b10313f4380b1b46f7b4e8641c6" candidate_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9089de23903d46ef92e1e327f664f42a"
I20260812 03:13:15.199231 21201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a88b10313f4380b1b46f7b4e8641c6" candidate_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c0ab645cc94a4eb4620fc74d17004e"
I20260812 03:13:15.199353 21480 raft_consensus.cc:3037] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:15.199353 21201 raft_consensus.cc:3037] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:15.200505 21480 raft_consensus.cc:2454] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13a38e66ae0c4c1ebd5f572e22bbb7fb in term 1.
I20260812 03:13:15.200505 21201 raft_consensus.cc:2454] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13a38e66ae0c4c1ebd5f572e22bbb7fb in term 1.
I20260812 03:13:15.200784 20786 leader_election.cc:302] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13a38e66ae0c4c1ebd5f572e22bbb7fb, b9c0ab645cc94a4eb4620fc74d17004e; no voters:
I20260812 03:13:15.200959 21620 raft_consensus.cc:2781] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:15.201157 21620 raft_consensus.cc:686] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 1 LEADER]: Becoming Leader. State: Replica: 13a38e66ae0c4c1ebd5f572e22bbb7fb, State: Running, Role: LEADER
I20260812 03:13:15.201522 21620 consensus_queue.cc:237] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:15.204010 21648 tablet_replica.cc:439] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: TxnStatusTablet state changed. Reason: New leader 13a38e66ae0c4c1ebd5f572e22bbb7fb. Latest consensus state: current_term: 1 leader_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } } }
I20260812 03:13:15.204149 21648 tablet_replica.cc:442] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:13:15.204460 21650 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 03:13:15.204670 21650 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 03:13:15.205140 20610 catalog_manager.cc:5495] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 13a38e66ae0c4c1ebd5f572e22bbb7fb (127.16.217.65). New cstate: current_term: 1 leader_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260812 03:13:15.252152 21619 consensus_queue.cc:1035] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [LEADER]: Connected to new peer: Peer: permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:15.257076 21622 consensus_queue.cc:1035] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:15.257490 21639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:15.252559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:37809 (local address 127.0.0.1:53912)
0812 03:13:15.252734 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:15.252752 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:15.252858 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:15.253532 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:15.253535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:15.253546 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:13:15.253681 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:15.253691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.254677 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.254681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:15.255540 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:15.255553 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.256185 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.256188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:15.256268 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:15.256321 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:13:15.257164 (+ 843us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:13:15.257179 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 03:13:15.257247 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:13:15.257334 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 03:13:15.257930 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:15.252846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53912 (local address 127.16.217.65:37809)
0812 03:13:15.252954 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:15.252958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:15.252971 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:13:15.253039 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:15.253042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:15.253118 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:15.253219 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:15.253820 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.254531 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.255689 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.255991 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.256093 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:15.256495 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:15.257007 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:15.257117 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:15.257757 (+ 640us) server_negotiation.cc:299] Negotiation successful
0812 03:13:15.257813 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:13:15.263479 21201 raft_consensus.cc:1261] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e [term 1 FOLLOWER]: Refusing update from remote peer 13a38e66ae0c4c1ebd5f572e22bbb7fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:15.263541 21480 raft_consensus.cc:1261] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a [term 1 FOLLOWER]: Refusing update from remote peer 13a38e66ae0c4c1ebd5f572e22bbb7fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:15.264087 21648 consensus_queue.cc:1035] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:15.264286 21620 consensus_queue.cc:1035] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:15.267004 21615 tablet_replica.cc:439] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e: TxnStatusTablet state changed. Reason: New leader 13a38e66ae0c4c1ebd5f572e22bbb7fb. Latest consensus state: current_term: 1 leader_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } } }
I20260812 03:13:15.267139 21615 tablet_replica.cc:442] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:15.268177 21622 tablet_replica.cc:439] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a: TxnStatusTablet state changed. Reason: New leader 13a38e66ae0c4c1ebd5f572e22bbb7fb. Latest consensus state: current_term: 1 leader_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } } }
I20260812 03:13:15.268293 21622 tablet_replica.cc:442] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:15.269062 21620 tablet_replica.cc:439] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } } }
I20260812 03:13:15.269172 21620 tablet_replica.cc:442] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:13:15.270375 21622 tablet_replica.cc:439] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } } }
I20260812 03:13:15.270483 21622 tablet_replica.cc:442] T 02a88b10313f4380b1b46f7b4e8641c6 P 9089de23903d46ef92e1e327f664f42a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:15.271260 21648 tablet_replica.cc:439] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } } }
I20260812 03:13:15.271369 21648 tablet_replica.cc:442] T 02a88b10313f4380b1b46f7b4e8641c6 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:13:15.272795 21615 tablet_replica.cc:439] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } } }
I20260812 03:13:15.272904 21615 tablet_replica.cc:442] T 02a88b10313f4380b1b46f7b4e8641c6 P b9c0ab645cc94a4eb4620fc74d17004e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:15.278697 21660 mvcc.cc:205] Tried to move back new op lower bound from 7317522002993442816 to 7317522002748018688. Current Snapshot: MvccSnapshot[applied={T|T < 7317522002993442816 or (T in {7317522002993442816})}]
I20260812 03:13:15.281183 21647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:15.277531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51318 (local address 127.16.217.67:44107)
0812 03:13:15.277641 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:15.277644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:15.277656 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:13:15.277717 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:15.277719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:15.277767 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:15.277882 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:15.278260 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.278891 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.279762 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.279990 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.280101 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:15.280373 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:15.280788 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:15.280911 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:15.280980 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 03:13:15.281035 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:13:15.292282 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:15.288606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:58573 (local address 127.16.217.65:37809)
0812 03:13:15.288745 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:15.288749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:15.288872 (+ 123us) server_negotiation.cc:407] Connection header received
0812 03:13:15.288998 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:15.289001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:15.289044 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:15.289121 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:15.289403 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.289912 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.291165 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.291384 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.291467 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:15.291819 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:15.291883 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:15.291936 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:15.292094 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 03:13:15.292157 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:13:15.292294 21682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:15.288436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:37809 (local address 127.16.217.67:58573)
0812 03:13:15.288779 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:15.288795 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:15.288902 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:15.289147 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:15.289151 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:15.289159 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:13:15.289264 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:15.289269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.290030 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.290033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:15.291024 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:15.291036 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.291515 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.291519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:15.291607 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:15.291661 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:13:15.291966 (+ 305us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:13:15.291973 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 03:13:15.292042 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 03:13:15.292127 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":123,"threads_started":1}
I20260812 03:13:15.319517 21647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:15.314992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:55313 (local address 127.16.217.67:44107)
0812 03:13:15.315276 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:15.315280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:15.315352 (+ 72us) server_negotiation.cc:407] Connection header received
0812 03:13:15.315517 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:15.315521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:15.315571 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:15.315661 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:15.316001 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.316810 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.318192 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.318515 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.318633 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:15.318991 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:15.319101 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:15.319147 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:15.319314 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 03:13:15.319372 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208}
I20260812 03:13:15.320005 21691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:15.314898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:44107 (local address 127.16.217.65:55313)
0812 03:13:15.315284 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:15.315300 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:15.315405 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:15.315673 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:15.315676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:15.315683 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:13:15.315827 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:15.315832 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.317010 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.317016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:15.318040 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:15.318052 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.318660 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.318664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:15.318763 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:15.318826 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:13:15.319216 (+ 390us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:13:15.319221 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:13:15.319749 (+ 528us) client_negotiation.cc:240] Negotiation successful
0812 03:13:15.319842 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":175,"threads_started":1}
I20260812 03:13:15.357381 20610 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:15.357594 20610 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:15.360601 20610 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc: Sending DeleteTablet for 3 replicas of tablet 0e1d1a308db3485296f5bc65275c6444
I20260812 03:13:15.361399 20877 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1d1a308db3485296f5bc65275c6444 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:15 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:53900
I20260812 03:13:15.361996 21163 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1d1a308db3485296f5bc65275c6444 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:15 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:58934
I20260812 03:13:15.362931 21699 tablet_replica.cc:330] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e: stopping tablet replica
I20260812 03:13:15.363394 21699 raft_consensus.cc:2229] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:15.363420 21446 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1d1a308db3485296f5bc65275c6444 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:15 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:51302
I20260812 03:13:15.363792 21699 raft_consensus.cc:2258] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:15.365296 21699 ts_tablet_manager.cc:1905] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:15.365242 20610 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:13:15.365732 20610 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:13:15.368857 21699 ts_tablet_manager.cc:1918] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:13:15.369089 21699 log.cc:1196] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/0e1d1a308db3485296f5bc65275c6444
I20260812 03:13:15.369505 21699 ts_tablet_manager.cc:1939] T 0e1d1a308db3485296f5bc65275c6444 P b9c0ab645cc94a4eb4620fc74d17004e: Deleting consensus metadata
I20260812 03:13:15.371058 20877 tablet_service.cc:1467] Processing CreateTablet for tablet f35f5fd56ba7413692b2a6c910fb3c2d (DEFAULT_TABLE table=test-table [id=036d0d647eb84d8695e93d814ca4f705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:15.371356 20877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f35f5fd56ba7413692b2a6c910fb3c2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:15.373747 21605 tablet_bootstrap.cc:492] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Bootstrap starting.
I20260812 03:13:15.373982 20558 catalog_manager.cc:4841] TS b9c0ab645cc94a4eb4620fc74d17004e (127.16.217.66:33493): tablet 0e1d1a308db3485296f5bc65275c6444 (table test-table [id=4b7b63fa0f9a4e4d89e6e941d45522fe]) successfully deleted
I20260812 03:13:15.374614 21605 tablet_bootstrap.cc:654] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:15.374768 21446 tablet_service.cc:1467] Processing CreateTablet for tablet f35f5fd56ba7413692b2a6c910fb3c2d (DEFAULT_TABLE table=test-table [id=036d0d647eb84d8695e93d814ca4f705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:15.375031 21446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f35f5fd56ba7413692b2a6c910fb3c2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:15.375581 21163 tablet_service.cc:1467] Processing CreateTablet for tablet f35f5fd56ba7413692b2a6c910fb3c2d (DEFAULT_TABLE table=test-table [id=036d0d647eb84d8695e93d814ca4f705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:15.375831 21163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f35f5fd56ba7413692b2a6c910fb3c2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:15.377540 21604 tablet_bootstrap.cc:492] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a: Bootstrap starting.
I20260812 03:13:15.378176 21607 tablet_bootstrap.cc:492] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e: Bootstrap starting.
I20260812 03:13:15.378530 21604 tablet_bootstrap.cc:654] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:15.379206 21607 tablet_bootstrap.cc:654] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:15.380739 21607 tablet_bootstrap.cc:492] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e: No bootstrap required, opened a new log
I20260812 03:13:15.380824 21607 ts_tablet_manager.cc:1397] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:15.381412 21607 raft_consensus.cc:348] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:15.381559 21607 raft_consensus.cc:374] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:15.381568 21604 tablet_bootstrap.cc:492] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a: No bootstrap required, opened a new log
I20260812 03:13:15.381596 21607 raft_consensus.cc:729] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9c0ab645cc94a4eb4620fc74d17004e, State: Initialized, Role: FOLLOWER
I20260812 03:13:15.381650 21604 ts_tablet_manager.cc:1397] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260812 03:13:15.381721 21607 consensus_queue.cc:260] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:15.381976 21607 ts_tablet_manager.cc:1428] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:15.382030 21604 raft_consensus.cc:348] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:15.382144 21604 raft_consensus.cc:374] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:15.382179 21604 raft_consensus.cc:729] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9089de23903d46ef92e1e327f664f42a, State: Initialized, Role: FOLLOWER
I20260812 03:13:15.382411 21700 tablet_replica.cc:330] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a: stopping tablet replica
I20260812 03:13:15.382722 21604 consensus_queue.cc:260] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:15.383083 21604 ts_tablet_manager.cc:1428] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:15.383967 21697 tablet_replica.cc:330] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: stopping tablet replica
I20260812 03:13:15.384303 21697 raft_consensus.cc:2229] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:15.384507 21697 raft_consensus.cc:2258] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:15.385066 21605 tablet_bootstrap.cc:492] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb: No bootstrap required, opened a new log
I20260812 03:13:15.385164 21605 ts_tablet_manager.cc:1397] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.001s
I20260812 03:13:15.385573 21605 raft_consensus.cc:348] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:15.385682 21605 raft_consensus.cc:374] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:15.385711 21605 raft_consensus.cc:729] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13a38e66ae0c4c1ebd5f572e22bbb7fb, State: Initialized, Role: FOLLOWER
I20260812 03:13:15.385834 21605 consensus_queue.cc:260] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:15.386106 21605 ts_tablet_manager.cc:1428] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:15.382732 21700 raft_consensus.cc:2229] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:13:15.386454 21697 ts_tablet_manager.cc:1905] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:15.386503 21700 raft_consensus.cc:2258] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:15.387706 21700 ts_tablet_manager.cc:1905] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:15.389557 21697 ts_tablet_manager.cc:1918] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:13:15.389678 21697 log.cc:1196] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/wal/wals/0e1d1a308db3485296f5bc65275c6444
I20260812 03:13:15.390015 21697 ts_tablet_manager.cc:1939] T 0e1d1a308db3485296f5bc65275c6444 P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Deleting consensus metadata
I20260812 03:13:15.390851 21700 ts_tablet_manager.cc:1918] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:13:15.390955 21700 log.cc:1196] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/0e1d1a308db3485296f5bc65275c6444
I20260812 03:13:15.391319 21700 ts_tablet_manager.cc:1939] T 0e1d1a308db3485296f5bc65275c6444 P 9089de23903d46ef92e1e327f664f42a: Deleting consensus metadata
I20260812 03:13:15.391350 20562 catalog_manager.cc:4841] TS 13a38e66ae0c4c1ebd5f572e22bbb7fb (127.16.217.65:37809): tablet 0e1d1a308db3485296f5bc65275c6444 (table test-table [id=4b7b63fa0f9a4e4d89e6e941d45522fe]) successfully deleted
I20260812 03:13:15.392320 20557 catalog_manager.cc:4841] TS 9089de23903d46ef92e1e327f664f42a (127.16.217.67:44107): tablet 0e1d1a308db3485296f5bc65275c6444 (table test-table [id=4b7b63fa0f9a4e4d89e6e941d45522fe]) successfully deleted
I20260812 03:13:15.469524 21671 raft_consensus.cc:482] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:15.469733 21671 raft_consensus.cc:504] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:15.470194 21671 leader_election.cc:288] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9c0ab645cc94a4eb4620fc74d17004e (127.16.217.66:33493), 9089de23903d46ef92e1e327f664f42a (127.16.217.67:44107)
I20260812 03:13:15.470670 21201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f35f5fd56ba7413692b2a6c910fb3c2d" candidate_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" is_pre_election: true
I20260812 03:13:15.470669 21480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f35f5fd56ba7413692b2a6c910fb3c2d" candidate_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9089de23903d46ef92e1e327f664f42a" is_pre_election: true
I20260812 03:13:15.470821 21480 raft_consensus.cc:2454] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13a38e66ae0c4c1ebd5f572e22bbb7fb in term 0.
I20260812 03:13:15.470821 21201 raft_consensus.cc:2454] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13a38e66ae0c4c1ebd5f572e22bbb7fb in term 0.
I20260812 03:13:15.471164 20785 leader_election.cc:302] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13a38e66ae0c4c1ebd5f572e22bbb7fb, 9089de23903d46ef92e1e327f664f42a; no voters:
I20260812 03:13:15.471382 21671 raft_consensus.cc:2781] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:13:15.471479 21671 raft_consensus.cc:482] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:13:15.471560 21671 raft_consensus.cc:3037] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:15.472606 21671 raft_consensus.cc:504] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:15.472987 21671 leader_election.cc:288] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [CANDIDATE]: Term 1 election: Requested vote from peers b9c0ab645cc94a4eb4620fc74d17004e (127.16.217.66:33493), 9089de23903d46ef92e1e327f664f42a (127.16.217.67:44107)
I20260812 03:13:15.473371 21201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f35f5fd56ba7413692b2a6c910fb3c2d" candidate_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c0ab645cc94a4eb4620fc74d17004e"
I20260812 03:13:15.473383 21480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f35f5fd56ba7413692b2a6c910fb3c2d" candidate_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9089de23903d46ef92e1e327f664f42a"
I20260812 03:13:15.473508 21201 raft_consensus.cc:3037] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:15.473515 21480 raft_consensus.cc:3037] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:15.474459 21201 raft_consensus.cc:2454] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13a38e66ae0c4c1ebd5f572e22bbb7fb in term 1.
I20260812 03:13:15.474746 21480 raft_consensus.cc:2454] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13a38e66ae0c4c1ebd5f572e22bbb7fb in term 1.
I20260812 03:13:15.474843 20786 leader_election.cc:302] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13a38e66ae0c4c1ebd5f572e22bbb7fb, b9c0ab645cc94a4eb4620fc74d17004e; no voters:
I20260812 03:13:15.475051 21671 raft_consensus.cc:2781] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:15.475163 21671 raft_consensus.cc:686] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 1 LEADER]: Becoming Leader. State: Replica: 13a38e66ae0c4c1ebd5f572e22bbb7fb, State: Running, Role: LEADER
I20260812 03:13:15.475340 21671 consensus_queue.cc:237] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:15.477522 20615 catalog_manager.cc:5495] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 13a38e66ae0c4c1ebd5f572e22bbb7fb (127.16.217.65). New cstate: current_term: 1 leader_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260812 03:13:15.497861 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:15.494460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53918 (local address 127.16.217.65:37809)
0812 03:13:15.494585 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:15.494590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:15.494606 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:13:15.494744 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:15.494747 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:15.494806 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:15.494896 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:15.495111 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.495838 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.496645 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.496971 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.497126 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:15.497419 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:15.497502 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:15.497586 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:15.497663 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 03:13:15.497721 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:13:15.507941 21691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:15.504147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:37809 (local address 127.16.217.65:42757)
0812 03:13:15.504357 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:15.504401 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:15.504487 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:15.504900 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:15.504903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:15.504910 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:13:15.504991 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:15.504995 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.505929 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.505932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:15.506672 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:15.506685 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.507280 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.507282 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:15.507343 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:15.507380 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:13:15.507689 (+ 309us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:13:15.507694 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:13:15.507750 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 03:13:15.507817 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260812 03:13:15.507961 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:15.504205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:42757 (local address 127.16.217.65:37809)
0812 03:13:15.504652 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:15.504657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:15.504668 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:13:15.504704 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:15.504722 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:15.504768 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:15.504843 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:15.505112 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.505809 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.506827 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:15.507145 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:15.507260 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:15.507512 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:15.507596 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:15.507679 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:15.507780 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 03:13:15.507832 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379}
I20260812 03:13:15.514300 21480 raft_consensus.cc:1261] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a [term 1 FOLLOWER]: Refusing update from remote peer 13a38e66ae0c4c1ebd5f572e22bbb7fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:15.514371 21201 raft_consensus.cc:1261] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e [term 1 FOLLOWER]: Refusing update from remote peer 13a38e66ae0c4c1ebd5f572e22bbb7fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:15.514761 21648 consensus_queue.cc:1035] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:15.514940 21620 consensus_queue.cc:1035] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:15.518769 21662 mvcc.cc:205] Tried to move back new op lower bound from 7317522004023214080 to 7317522003868626944. Current Snapshot: MvccSnapshot[applied={T|T < 7317522004023214080}]
I20260812 03:13:15.519577 21657 mvcc.cc:205] Tried to move back new op lower bound from 7317522004023214080 to 7317522003868626944. Current Snapshot: MvccSnapshot[applied={T|T < 7317522004023214080}]
W20260812 03:13:15.531463 21662 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 03:13:17.104450 20610 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:17.104640 20610 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:17.107568 20610 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc: Sending DeleteTablet for 3 replicas of tablet f35f5fd56ba7413692b2a6c910fb3c2d
I20260812 03:13:17.108171 20877 tablet_service.cc:1514] Processing DeleteTablet for tablet f35f5fd56ba7413692b2a6c910fb3c2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:17 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:53900
I20260812 03:13:17.108211 21163 tablet_service.cc:1514] Processing DeleteTablet for tablet f35f5fd56ba7413692b2a6c910fb3c2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:17 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:58934
I20260812 03:13:17.108296 21446 tablet_service.cc:1514] Processing DeleteTablet for tablet f35f5fd56ba7413692b2a6c910fb3c2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:17 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:51302
I20260812 03:13:17.108541 21746 tablet_replica.cc:330] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb: stopping tablet replica
I20260812 03:13:17.108690 21746 raft_consensus.cc:2229] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:13:17.108894 21750 tablet_replica.cc:330] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e: stopping tablet replica
I20260812 03:13:17.109021 21752 tablet_replica.cc:330] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a: stopping tablet replica
I20260812 03:13:17.109072 21746 raft_consensus.cc:2258] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:17.109246 21750 raft_consensus.cc:2229] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:17.109431 21752 raft_consensus.cc:2229] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:17.109616 21750 raft_consensus.cc:2258] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:17.109800 21752 raft_consensus.cc:2258] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:17.109803 21746 ts_tablet_manager.cc:1905] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:17.110625 21750 ts_tablet_manager.cc:1905] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:17.113184 21752 ts_tablet_manager.cc:1905] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:17.113411 21746 ts_tablet_manager.cc:1918] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:13:17.113433 21750 ts_tablet_manager.cc:1918] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:13:17.113505 21746 log.cc:1196] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/wal/wals/f35f5fd56ba7413692b2a6c910fb3c2d
I20260812 03:13:17.113611 21750 log.cc:1196] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/f35f5fd56ba7413692b2a6c910fb3c2d
I20260812 03:13:17.113847 21746 ts_tablet_manager.cc:1939] T f35f5fd56ba7413692b2a6c910fb3c2d P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Deleting consensus metadata
I20260812 03:13:17.113981 21750 ts_tablet_manager.cc:1939] T f35f5fd56ba7413692b2a6c910fb3c2d P b9c0ab645cc94a4eb4620fc74d17004e: Deleting consensus metadata
I20260812 03:13:17.114722 20562 catalog_manager.cc:4841] TS 13a38e66ae0c4c1ebd5f572e22bbb7fb (127.16.217.65:37809): tablet f35f5fd56ba7413692b2a6c910fb3c2d (table test-table [id=036d0d647eb84d8695e93d814ca4f705]) successfully deleted
I20260812 03:13:17.114722 20558 catalog_manager.cc:4841] TS b9c0ab645cc94a4eb4620fc74d17004e (127.16.217.66:33493): tablet f35f5fd56ba7413692b2a6c910fb3c2d (table test-table [id=036d0d647eb84d8695e93d814ca4f705]) successfully deleted
I20260812 03:13:17.115909 21752 ts_tablet_manager.cc:1918] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:13:17.115989 21752 log.cc:1196] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/f35f5fd56ba7413692b2a6c910fb3c2d
I20260812 03:13:17.116232 21752 ts_tablet_manager.cc:1939] T f35f5fd56ba7413692b2a6c910fb3c2d P 9089de23903d46ef92e1e327f664f42a: Deleting consensus metadata
I20260812 03:13:17.116978 20557 catalog_manager.cc:4841] TS 9089de23903d46ef92e1e327f664f42a (127.16.217.67:44107): tablet f35f5fd56ba7413692b2a6c910fb3c2d (table test-table [id=036d0d647eb84d8695e93d814ca4f705]) successfully deleted
Aug 12 03:13:17 dist-test-slave-7nm7 krb5kdc[20496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504397, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:13:17 dist-test-slave-7nm7 krb5kdc[20496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504397, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:17.143559 21768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:17.135113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54120 (local address 127.16.217.126:43383)
0812 03:13:17.135441 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:17.135448 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:17.135463 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:13:17.135546 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:17.135548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:17.135591 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:17.135673 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:17.136364 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:17.136902 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:17.137707 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:17.137927 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:17.138007 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:17.140389 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:17.140393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:17.140395 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:17.140418 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:17.141730 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:17.142169 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:17.142175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:17.142181 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:17.142347 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:17.142632 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:17.142635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:17.142637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:17.142830 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:17.142977 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:17.143242 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 03:13:17.143371 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":129,"threads_started":1}
I20260812 03:13:17.146306 20610 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:13:17.146777 20610 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:13:17.151629 21163 tablet_service.cc:1467] Processing CreateTablet for tablet de9cab01b18b45f8a5df07edea5761ab (DEFAULT_TABLE table=test-table [id=f774dd103fa540589b8270749c11b8ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:17.151650 20877 tablet_service.cc:1467] Processing CreateTablet for tablet de9cab01b18b45f8a5df07edea5761ab (DEFAULT_TABLE table=test-table [id=f774dd103fa540589b8270749c11b8ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:17.151724 21446 tablet_service.cc:1467] Processing CreateTablet for tablet de9cab01b18b45f8a5df07edea5761ab (DEFAULT_TABLE table=test-table [id=f774dd103fa540589b8270749c11b8ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:17.151937 21163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de9cab01b18b45f8a5df07edea5761ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:17.151964 21446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de9cab01b18b45f8a5df07edea5761ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:17.151964 20877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de9cab01b18b45f8a5df07edea5761ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:17.153957 21772 tablet_bootstrap.cc:492] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e: Bootstrap starting.
I20260812 03:13:17.154592 21777 tablet_bootstrap.cc:492] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a: Bootstrap starting.
I20260812 03:13:17.155512 21777 tablet_bootstrap.cc:654] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:17.155522 21772 tablet_bootstrap.cc:654] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:17.156150 21778 tablet_bootstrap.cc:492] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Bootstrap starting.
I20260812 03:13:17.156704 21777 tablet_bootstrap.cc:492] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a: No bootstrap required, opened a new log
I20260812 03:13:17.156780 21777 ts_tablet_manager.cc:1397] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:13:17.156992 21772 tablet_bootstrap.cc:492] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e: No bootstrap required, opened a new log
I20260812 03:13:17.157097 21772 ts_tablet_manager.cc:1397] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:17.157238 21777 raft_consensus.cc:348] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:17.157369 21777 raft_consensus.cc:374] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:17.157413 21777 raft_consensus.cc:729] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9089de23903d46ef92e1e327f664f42a, State: Initialized, Role: FOLLOWER
I20260812 03:13:17.157559 21777 consensus_queue.cc:260] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:17.157610 21772 raft_consensus.cc:348] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:17.157732 21772 raft_consensus.cc:374] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:17.157773 21772 raft_consensus.cc:729] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9c0ab645cc94a4eb4620fc74d17004e, State: Initialized, Role: FOLLOWER
I20260812 03:13:17.157946 21772 consensus_queue.cc:260] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:17.158092 21777 ts_tablet_manager.cc:1428] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 03:13:17.158454 21772 ts_tablet_manager.cc:1428] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 03:13:17.159586 21778 tablet_bootstrap.cc:654] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:17.161078 21778 tablet_bootstrap.cc:492] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb: No bootstrap required, opened a new log
I20260812 03:13:17.161191 21778 ts_tablet_manager.cc:1397] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 03:13:17.161695 21778 raft_consensus.cc:348] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:17.161794 21778 raft_consensus.cc:374] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:17.161816 21778 raft_consensus.cc:729] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13a38e66ae0c4c1ebd5f572e22bbb7fb, State: Initialized, Role: FOLLOWER
I20260812 03:13:17.161973 21778 consensus_queue.cc:260] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:17.162308 21778 ts_tablet_manager.cc:1428] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 03:13:17.258150 21781 raft_consensus.cc:482] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:17.258351 21781 raft_consensus.cc:504] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:17.258829 21781 leader_election.cc:288] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13a38e66ae0c4c1ebd5f572e22bbb7fb (127.16.217.65:37809), b9c0ab645cc94a4eb4620fc74d17004e (127.16.217.66:33493)
I20260812 03:13:17.259353 21201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de9cab01b18b45f8a5df07edea5761ab" candidate_uuid: "9089de23903d46ef92e1e327f664f42a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" is_pre_election: true
I20260812 03:13:17.259353 20911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de9cab01b18b45f8a5df07edea5761ab" candidate_uuid: "9089de23903d46ef92e1e327f664f42a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" is_pre_election: true
I20260812 03:13:17.259519 21201 raft_consensus.cc:2454] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9089de23903d46ef92e1e327f664f42a in term 0.
I20260812 03:13:17.259519 20911 raft_consensus.cc:2454] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9089de23903d46ef92e1e327f664f42a in term 0.
I20260812 03:13:17.259892 21340 leader_election.cc:302] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13a38e66ae0c4c1ebd5f572e22bbb7fb, 9089de23903d46ef92e1e327f664f42a; no voters:
I20260812 03:13:17.260108 21781 raft_consensus.cc:2781] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:13:17.260185 21781 raft_consensus.cc:482] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:13:17.260228 21781 raft_consensus.cc:3037] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:17.261312 21781 raft_consensus.cc:504] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:17.261685 21781 leader_election.cc:288] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [CANDIDATE]: Term 1 election: Requested vote from peers 13a38e66ae0c4c1ebd5f572e22bbb7fb (127.16.217.65:37809), b9c0ab645cc94a4eb4620fc74d17004e (127.16.217.66:33493)
I20260812 03:13:17.262075 20911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de9cab01b18b45f8a5df07edea5761ab" candidate_uuid: "9089de23903d46ef92e1e327f664f42a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb"
I20260812 03:13:17.262156 21201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de9cab01b18b45f8a5df07edea5761ab" candidate_uuid: "9089de23903d46ef92e1e327f664f42a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c0ab645cc94a4eb4620fc74d17004e"
I20260812 03:13:17.262217 20911 raft_consensus.cc:3037] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:17.262341 21201 raft_consensus.cc:3037] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:17.263306 20911 raft_consensus.cc:2454] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9089de23903d46ef92e1e327f664f42a in term 1.
I20260812 03:13:17.263521 21201 raft_consensus.cc:2454] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9089de23903d46ef92e1e327f664f42a in term 1.
I20260812 03:13:17.263670 21340 leader_election.cc:302] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13a38e66ae0c4c1ebd5f572e22bbb7fb, 9089de23903d46ef92e1e327f664f42a; no voters:
I20260812 03:13:17.263876 21781 raft_consensus.cc:2781] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:17.264127 21781 raft_consensus.cc:686] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [term 1 LEADER]: Becoming Leader. State: Replica: 9089de23903d46ef92e1e327f664f42a, State: Running, Role: LEADER
I20260812 03:13:17.264307 21781 consensus_queue.cc:237] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:17.266119 20615 catalog_manager.cc:5495] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9089de23903d46ef92e1e327f664f42a (127.16.217.67). New cstate: current_term: 1 leader_uuid: "9089de23903d46ef92e1e327f664f42a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260812 03:13:17.292044 21798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:17.288754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51330 (local address 127.16.217.67:44107)
0812 03:13:17.289032 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:17.289041 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:17.289061 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:13:17.289098 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:17.289100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:17.289140 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:17.289250 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:17.289494 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:17.290028 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:17.290956 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:17.291197 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:17.291313 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:17.291569 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:17.291649 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:17.291734 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:17.291828 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 03:13:17.291883 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":98,"threads_started":1}
I20260812 03:13:17.298532 21201 raft_consensus.cc:1261] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e [term 1 FOLLOWER]: Refusing update from remote peer 9089de23903d46ef92e1e327f664f42a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:17.298657 20911 raft_consensus.cc:1261] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 1 FOLLOWER]: Refusing update from remote peer 9089de23903d46ef92e1e327f664f42a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:17.298954 21791 consensus_queue.cc:1035] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:17.299386 21781 consensus_queue.cc:1035] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [LEADER]: Connected to new peer: Peer: permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:17.331576 20606 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:17.331732 20606 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:17.334008 20606 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc: Sending DeleteTablet for 3 replicas of tablet de9cab01b18b45f8a5df07edea5761ab
I20260812 03:13:17.334756 21446 tablet_service.cc:1514] Processing DeleteTablet for tablet de9cab01b18b45f8a5df07edea5761ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:17 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:51302
I20260812 03:13:17.334797 20877 tablet_service.cc:1514] Processing DeleteTablet for tablet de9cab01b18b45f8a5df07edea5761ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:17 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:53900
I20260812 03:13:17.334802 21163 tablet_service.cc:1514] Processing DeleteTablet for tablet de9cab01b18b45f8a5df07edea5761ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:17 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:58934
I20260812 03:13:17.334969 21752 tablet_replica.cc:330] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a: stopping tablet replica
I20260812 03:13:17.335006 21750 tablet_replica.cc:330] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e: stopping tablet replica
I20260812 03:13:17.335022 21746 tablet_replica.cc:330] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb: stopping tablet replica
I20260812 03:13:17.335093 21752 raft_consensus.cc:2229] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:13:17.335106 21750 raft_consensus.cc:2229] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:17.335160 21746 raft_consensus.cc:2229] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:17.335207 21750 raft_consensus.cc:2258] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:17.335301 21752 raft_consensus.cc:2258] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:17.335307 21746 raft_consensus.cc:2258] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:17.335486 20606 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:13:17.335945 20606 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:13:17.336220 21752 ts_tablet_manager.cc:1905] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:17.336220 21746 ts_tablet_manager.cc:1905] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:17.336525 21750 ts_tablet_manager.cc:1905] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:17.338796 21746 ts_tablet_manager.cc:1918] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:13:17.338796 21752 ts_tablet_manager.cc:1918] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:13:17.338877 21752 log.cc:1196] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/de9cab01b18b45f8a5df07edea5761ab
I20260812 03:13:17.338877 21746 log.cc:1196] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-0/wal/wals/de9cab01b18b45f8a5df07edea5761ab
I20260812 03:13:17.339172 21752 ts_tablet_manager.cc:1939] T de9cab01b18b45f8a5df07edea5761ab P 9089de23903d46ef92e1e327f664f42a: Deleting consensus metadata
I20260812 03:13:17.339172 21746 ts_tablet_manager.cc:1939] T de9cab01b18b45f8a5df07edea5761ab P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Deleting consensus metadata
I20260812 03:13:17.339790 21750 ts_tablet_manager.cc:1918] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:13:17.339876 21750 log.cc:1196] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/de9cab01b18b45f8a5df07edea5761ab
I20260812 03:13:17.339908 20557 catalog_manager.cc:4841] TS 9089de23903d46ef92e1e327f664f42a (127.16.217.67:44107): tablet de9cab01b18b45f8a5df07edea5761ab (table test-table [id=f774dd103fa540589b8270749c11b8ca]) successfully deleted
I20260812 03:13:17.340131 20562 catalog_manager.cc:4841] TS 13a38e66ae0c4c1ebd5f572e22bbb7fb (127.16.217.65:37809): tablet de9cab01b18b45f8a5df07edea5761ab (table test-table [id=f774dd103fa540589b8270749c11b8ca]) successfully deleted
I20260812 03:13:17.340179 21750 ts_tablet_manager.cc:1939] T de9cab01b18b45f8a5df07edea5761ab P b9c0ab645cc94a4eb4620fc74d17004e: Deleting consensus metadata
I20260812 03:13:17.341012 20558 catalog_manager.cc:4841] TS b9c0ab645cc94a4eb4620fc74d17004e (127.16.217.66:33493): tablet de9cab01b18b45f8a5df07edea5761ab (table test-table [id=f774dd103fa540589b8270749c11b8ca]) successfully deleted
I20260812 03:13:17.343720 21163 tablet_service.cc:1467] Processing CreateTablet for tablet 14206a3ec7d540b9abf98476d0657beb (DEFAULT_TABLE table=test-table [id=d9d36d1b31a648e4b662652bbef45eb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:17.343736 20877 tablet_service.cc:1467] Processing CreateTablet for tablet 14206a3ec7d540b9abf98476d0657beb (DEFAULT_TABLE table=test-table [id=d9d36d1b31a648e4b662652bbef45eb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:17.344015 21163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14206a3ec7d540b9abf98476d0657beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:17.344012 20877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14206a3ec7d540b9abf98476d0657beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:17.346045 21772 tablet_bootstrap.cc:492] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e: Bootstrap starting.
I20260812 03:13:17.346467 21446 tablet_service.cc:1467] Processing CreateTablet for tablet 14206a3ec7d540b9abf98476d0657beb (DEFAULT_TABLE table=test-table [id=d9d36d1b31a648e4b662652bbef45eb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:17.346755 21446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14206a3ec7d540b9abf98476d0657beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:17.347050 21772 tablet_bootstrap.cc:654] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:17.348203 21772 tablet_bootstrap.cc:492] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e: No bootstrap required, opened a new log
I20260812 03:13:17.348316 21772 ts_tablet_manager.cc:1397] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:13:17.348690 21772 raft_consensus.cc:348] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:17.348803 21772 raft_consensus.cc:374] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:17.348845 21772 raft_consensus.cc:729] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9c0ab645cc94a4eb4620fc74d17004e, State: Initialized, Role: FOLLOWER
I20260812 03:13:17.348863 21777 tablet_bootstrap.cc:492] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a: Bootstrap starting.
I20260812 03:13:17.349040 21778 tablet_bootstrap.cc:492] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Bootstrap starting.
I20260812 03:13:17.348989 21772 consensus_queue.cc:260] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:17.349282 21772 ts_tablet_manager.cc:1428] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:13:17.349686 21777 tablet_bootstrap.cc:654] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:17.349807 21778 tablet_bootstrap.cc:654] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:17.350924 21777 tablet_bootstrap.cc:492] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a: No bootstrap required, opened a new log
I20260812 03:13:17.351017 21777 ts_tablet_manager.cc:1397] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:13:17.351099 21778 tablet_bootstrap.cc:492] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb: No bootstrap required, opened a new log
I20260812 03:13:17.351163 21778 ts_tablet_manager.cc:1397] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:13:17.351328 21777 raft_consensus.cc:348] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:17.351405 21777 raft_consensus.cc:374] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:17.351426 21777 raft_consensus.cc:729] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9089de23903d46ef92e1e327f664f42a, State: Initialized, Role: FOLLOWER
I20260812 03:13:17.351477 21778 raft_consensus.cc:348] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:17.351557 21778 raft_consensus.cc:374] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:17.351576 21778 raft_consensus.cc:729] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13a38e66ae0c4c1ebd5f572e22bbb7fb, State: Initialized, Role: FOLLOWER
I20260812 03:13:17.351559 21777 consensus_queue.cc:260] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:17.351712 21778 consensus_queue.cc:260] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:17.351842 21777 ts_tablet_manager.cc:1428] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:13:17.352005 21778 ts_tablet_manager.cc:1428] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:13:17.458323 21713 raft_consensus.cc:482] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:17.458505 21713 raft_consensus.cc:504] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:17.458966 21713 leader_election.cc:288] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9c0ab645cc94a4eb4620fc74d17004e (127.16.217.66:33493), 9089de23903d46ef92e1e327f664f42a (127.16.217.67:44107)
I20260812 03:13:17.459506 21480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14206a3ec7d540b9abf98476d0657beb" candidate_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9089de23903d46ef92e1e327f664f42a" is_pre_election: true
I20260812 03:13:17.459515 21201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14206a3ec7d540b9abf98476d0657beb" candidate_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" is_pre_election: true
I20260812 03:13:17.459681 21480 raft_consensus.cc:2454] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13a38e66ae0c4c1ebd5f572e22bbb7fb in term 0.
I20260812 03:13:17.459684 21201 raft_consensus.cc:2454] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13a38e66ae0c4c1ebd5f572e22bbb7fb in term 0.
I20260812 03:13:17.460006 20786 leader_election.cc:302] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13a38e66ae0c4c1ebd5f572e22bbb7fb, b9c0ab645cc94a4eb4620fc74d17004e; no voters:
I20260812 03:13:17.460255 21713 raft_consensus.cc:2781] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:13:17.460325 21713 raft_consensus.cc:482] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:13:17.460353 21713 raft_consensus.cc:3037] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:17.461442 21713 raft_consensus.cc:504] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:17.461799 21713 leader_election.cc:288] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [CANDIDATE]: Term 1 election: Requested vote from peers b9c0ab645cc94a4eb4620fc74d17004e (127.16.217.66:33493), 9089de23903d46ef92e1e327f664f42a (127.16.217.67:44107)
I20260812 03:13:17.462177 21480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14206a3ec7d540b9abf98476d0657beb" candidate_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9089de23903d46ef92e1e327f664f42a"
I20260812 03:13:17.462193 21201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14206a3ec7d540b9abf98476d0657beb" candidate_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c0ab645cc94a4eb4620fc74d17004e"
I20260812 03:13:17.462357 21480 raft_consensus.cc:3037] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:17.462389 21201 raft_consensus.cc:3037] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:17.463416 21201 raft_consensus.cc:2454] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13a38e66ae0c4c1ebd5f572e22bbb7fb in term 1.
I20260812 03:13:17.463418 21480 raft_consensus.cc:2454] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13a38e66ae0c4c1ebd5f572e22bbb7fb in term 1.
I20260812 03:13:17.463747 20786 leader_election.cc:302] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13a38e66ae0c4c1ebd5f572e22bbb7fb, b9c0ab645cc94a4eb4620fc74d17004e; no voters:
I20260812 03:13:17.463905 21713 raft_consensus.cc:2781] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:17.464003 21713 raft_consensus.cc:686] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [term 1 LEADER]: Becoming Leader. State: Replica: 13a38e66ae0c4c1ebd5f572e22bbb7fb, State: Running, Role: LEADER
I20260812 03:13:17.464157 21713 consensus_queue.cc:237] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } }
I20260812 03:13:17.465988 20615 catalog_manager.cc:5495] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 13a38e66ae0c4c1ebd5f572e22bbb7fb (127.16.217.65). New cstate: current_term: 1 leader_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a38e66ae0c4c1ebd5f572e22bbb7fb" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 37809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260812 03:13:17.508750 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:17.505734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53930 (local address 127.16.217.65:37809)
0812 03:13:17.506039 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:17.506048 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:17.506061 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:13:17.506099 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:17.506101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:17.506142 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:17.506222 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:17.506476 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:17.506987 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:17.507792 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:17.508004 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:17.508080 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:17.508307 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:17.508374 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:17.508429 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:17.508509 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 03:13:17.508553 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":144,"threads_started":1}
I20260812 03:13:17.515282 21201 raft_consensus.cc:1261] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e [term 1 FOLLOWER]: Refusing update from remote peer 13a38e66ae0c4c1ebd5f572e22bbb7fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:17.515383 21480 raft_consensus.cc:1261] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a [term 1 FOLLOWER]: Refusing update from remote peer 13a38e66ae0c4c1ebd5f572e22bbb7fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:17.515668 21713 consensus_queue.cc:1035] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9c0ab645cc94a4eb4620fc74d17004e" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:17.515807 21620 consensus_queue.cc:1035] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9089de23903d46ef92e1e327f664f42a" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:13:17.530278 21711 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 03:13:19.083698 20606 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:19.083873 20606 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:19.086220 20606 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca598746d4ce4dc19f8d928aefa19acc: Sending DeleteTablet for 3 replicas of tablet 14206a3ec7d540b9abf98476d0657beb
I20260812 03:13:19.086887 20877 tablet_service.cc:1514] Processing DeleteTablet for tablet 14206a3ec7d540b9abf98476d0657beb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:19 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:53900
I20260812 03:13:19.086935 21163 tablet_service.cc:1514] Processing DeleteTablet for tablet 14206a3ec7d540b9abf98476d0657beb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:19 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:58934
I20260812 03:13:19.086998 21446 tablet_service.cc:1514] Processing DeleteTablet for tablet 14206a3ec7d540b9abf98476d0657beb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:19 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:51302
I20260812 03:13:19.087257 21849 tablet_replica.cc:330] T 14206a3ec7d540b9abf98476d0657beb P 13a38e66ae0c4c1ebd5f572e22bbb7fb: stopping tablet replica
I20260812 03:13:19.087328 21850 tablet_replica.cc:330] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e: stopping tablet replica
I20260812 03:13:19.087319 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 20751
I20260812 03:13:19.087360 21852 tablet_replica.cc:330] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a: stopping tablet replica
I20260812 03:13:19.104465 21852 raft_consensus.cc:2229] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:19.104808 21852 raft_consensus.cc:2258] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:19.106877 21852 ts_tablet_manager.cc:1905] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:19.111086 21852 ts_tablet_manager.cc:1918] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:13:19.111284 21852 log.cc:1196] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/14206a3ec7d540b9abf98476d0657beb
I20260812 03:13:19.111739 21852 ts_tablet_manager.cc:1939] T 14206a3ec7d540b9abf98476d0657beb P 9089de23903d46ef92e1e327f664f42a: Deleting consensus metadata
I20260812 03:13:19.113762 20557 catalog_manager.cc:4841] TS 9089de23903d46ef92e1e327f664f42a (127.16.217.67:44107): tablet 14206a3ec7d540b9abf98476d0657beb (table test-table [id=d9d36d1b31a648e4b662652bbef45eb0]) successfully deleted
I20260812 03:13:19.114975 21850 raft_consensus.cc:2229] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:19.115183 21850 raft_consensus.cc:2258] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:19.116344 21850 ts_tablet_manager.cc:1905] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:19.119415 21850 ts_tablet_manager.cc:1918] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:13:19.119508 21850 log.cc:1196] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/14206a3ec7d540b9abf98476d0657beb
I20260812 03:13:19.119799 21850 ts_tablet_manager.cc:1939] T 14206a3ec7d540b9abf98476d0657beb P b9c0ab645cc94a4eb4620fc74d17004e: Deleting consensus metadata
I20260812 03:13:19.120612 20558 catalog_manager.cc:4841] TS b9c0ab645cc94a4eb4620fc74d17004e (127.16.217.66:33493): tablet 14206a3ec7d540b9abf98476d0657beb (table test-table [id=d9d36d1b31a648e4b662652bbef45eb0]) successfully deleted
W20260812 03:13:19.127174 20562 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.217.65:37809 (error 108)
I20260812 03:13:19.128068 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 21024
I20260812 03:13:19.130873 21854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:19.127908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:37809 (local address 127.0.0.1:53942)
0812 03:13:19.130574 (+ 2666us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:19.130712 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:37809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":83,"threads_started":1}
W20260812 03:13:19.131204 20562 catalog_manager.cc:4584] TS 13a38e66ae0c4c1ebd5f572e22bbb7fb (127.16.217.65:37809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 14206a3ec7d540b9abf98476d0657beb: Network error: Client connection negotiation failed: client connection to 127.16.217.65:37809: connect: Connection refused (error 111)
I20260812 03:13:19.147701 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 21311
I20260812 03:13:19.170158 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 20526
2026-08-12T03:13:19Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7061 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:13:19 dist-test-slave-7nm7 krb5kdc[21859](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:13:19 dist-test-slave-7nm7 krb5kdc[21859](info): set up 2 sockets
Aug 12 03:13:19 dist-test-slave-7nm7 krb5kdc[21859](info): commencing operation
krb5kdc: starting...
W20260812 03:13:20.520845 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:13:20 dist-test-slave-7nm7 krb5kdc[21859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:13:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:13:20.647243 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:37221
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:34787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:37221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:20.757716 21887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:20.758042 21887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:20.758119 21887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:20.762126 21887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:20.762233 21887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:20.762293 21887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:20.762333 21887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:20.762369 21887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:13:20.767702 21887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:34787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:37221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:37221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:20.768800 21887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:20.769848 21887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:13:20.778823 21887 server_base.cc:1034] running on GCE node
W20260812 03:13:20.778769 21895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:20.778760 21900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:20.778786 21903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:20.779737 21887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:20.780858 21887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:20.782061 21887 hybrid_clock.cc:648] HybridClock initialized: now 1786504400782038 us; error 34 us; skew 500 ppm
Aug 12 03:13:20 dist-test-slave-7nm7 krb5kdc[21859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504400, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:20.785266 21887 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:13:20.786311 21887 webserver.cc:460] Webserver started at http://127.16.217.126:33621/ using document root <none> and password file <none>
I20260812 03:13:20.786895 21887 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:20.786988 21887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:20.787248 21887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:20.789330 21887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "de456b630bbc4604b23203a8feb3981f"
format_stamp: "Formatted at 2026-08-12 03:13:20 on dist-test-slave-7nm7"
server_key: "217646f3fb950588500b1595854aea57"
server_key_iv: "30f434a87f53bd318b59a148d6bdfd49"
server_key_version: "encryptionkey@0"
I20260812 03:13:20.789909 21887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "de456b630bbc4604b23203a8feb3981f"
format_stamp: "Formatted at 2026-08-12 03:13:20 on dist-test-slave-7nm7"
server_key: "217646f3fb950588500b1595854aea57"
server_key_iv: "30f434a87f53bd318b59a148d6bdfd49"
server_key_version: "encryptionkey@0"
I20260812 03:13:20.793821 21887 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:13:20.796387 21915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:20.797631 21887 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:13:20.797778 21887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "de456b630bbc4604b23203a8feb3981f"
format_stamp: "Formatted at 2026-08-12 03:13:20 on dist-test-slave-7nm7"
server_key: "217646f3fb950588500b1595854aea57"
server_key_iv: "30f434a87f53bd318b59a148d6bdfd49"
server_key_version: "encryptionkey@0"
I20260812 03:13:20.797895 21887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:20.830153 21887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:20.833420 21887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:20.833668 21887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:20.841585 21887 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:37221
I20260812 03:13:20.841611 22011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:37221 every 8 connection(s)
I20260812 03:13:20.842653 21887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:20.844291 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 21887
I20260812 03:13:20.844401 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:13:20.844697 17253 external_mini_cluster.cc:1428] Setting key 0b5c6cd9d1bf2fa27a213fbfaf60c07d
I20260812 03:13:20.846148 22012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:20.853133 22012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f: Bootstrap starting.
Aug 12 03:13:20 dist-test-slave-7nm7 krb5kdc[21859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:20.856237 22012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:20.857333 22012 log.cc:824] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:20.860177 22012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f: No bootstrap required, opened a new log
I20260812 03:13:20.861526 22019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:20.846219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50124 (local address 127.16.217.126:37221)
0812 03:13:20.846708 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:20.846725 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:20.846761 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:13:20.847389 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:20.847405 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:20.847732 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:20.848007 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:20.848905 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:20.850073 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:20.851295 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:20.851788 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:20.851916 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:20.855303 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:20.855309 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:20.855326 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:20.855363 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:20.857616 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:20.858184 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:20.858189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:20.858193 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:20.858426 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:20.858810 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:20.858814 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:20.858815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:20.859495 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:20.859741 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:20.859977 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 03:13:20.860536 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":146,"threads_started":1}
I20260812 03:13:20.864646 22012 raft_consensus.cc:348] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de456b630bbc4604b23203a8feb3981f" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 37221 } }
I20260812 03:13:20.864889 22012 raft_consensus.cc:374] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:20.864926 22012 raft_consensus.cc:729] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de456b630bbc4604b23203a8feb3981f, State: Initialized, Role: FOLLOWER
I20260812 03:13:20.865298 22012 consensus_queue.cc:260] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de456b630bbc4604b23203a8feb3981f" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 37221 } }
I20260812 03:13:20.865412 22012 raft_consensus.cc:388] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:20.865453 22012 raft_consensus.cc:482] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:20.865528 22012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:20.866572 22012 raft_consensus.cc:504] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de456b630bbc4604b23203a8feb3981f" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 37221 } }
I20260812 03:13:20.866888 22012 leader_election.cc:302] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de456b630bbc4604b23203a8feb3981f; no voters:
I20260812 03:13:20.867177 22012 leader_election.cc:288] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:13:20.867360 22021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:20.867633 22021 raft_consensus.cc:686] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f [term 1 LEADER]: Becoming Leader. State: Replica: de456b630bbc4604b23203a8feb3981f, State: Running, Role: LEADER
I20260812 03:13:20.867966 22021 consensus_queue.cc:237] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de456b630bbc4604b23203a8feb3981f" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 37221 } }
I20260812 03:13:20.868512 22012 sys_catalog.cc:564] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:20.869899 22022 sys_catalog.cc:455] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de456b630bbc4604b23203a8feb3981f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de456b630bbc4604b23203a8feb3981f" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 37221 } } }
I20260812 03:13:20.870026 22022 sys_catalog.cc:458] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:20.870344 22028 sys_catalog.cc:455] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f [sys.catalog]: SysCatalogTable state changed. Reason: New leader de456b630bbc4604b23203a8feb3981f. Latest consensus state: current_term: 1 leader_uuid: "de456b630bbc4604b23203a8feb3981f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de456b630bbc4604b23203a8feb3981f" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 37221 } } }
I20260812 03:13:20.870464 22028 sys_catalog.cc:458] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:20.870625 22042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:20.874212 22042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:20.880153 22042 catalog_manager.cc:1347] Generated new cluster ID: fa39620e9f75440ea687f72ff26ee26d
I20260812 03:13:20.880256 22042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:20.899088 22042 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:13:20.901684 22042 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:20.913303 22042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de456b630bbc4604b23203a8feb3981f: Generated new TSK 0
I20260812 03:13:20.914283 22042 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
I20260812 03:13:20.960058 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:37221
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:34787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:21.082033 22064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:21.082392 22064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:21.082494 22064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:21.086607 22064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:21.086715 22064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:21.086817 22064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:13:21.092253 22064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:34787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:37221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:21.093366 22064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:21.094470 22064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:21.104588 22075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:21.104568 22069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:21.104568 22070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:21.104811 22064 server_base.cc:1034] running on GCE node
I20260812 03:13:21.105312 22064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:21.106061 22064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:21.107270 22064 hybrid_clock.cc:648] HybridClock initialized: now 1786504401107252 us; error 33 us; skew 500 ppm
Aug 12 03:13:21 dist-test-slave-7nm7 krb5kdc[21859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504401, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:21.110132 22064 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:13:21.111145 22064 webserver.cc:460] Webserver started at http://127.16.217.65:32999/ using document root <none> and password file <none>
I20260812 03:13:21.111809 22064 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:21.111912 22064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:21.112174 22064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:21.114351 22064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "3f70e7a14b1f4043ab810eee6b012765"
format_stamp: "Formatted at 2026-08-12 03:13:21 on dist-test-slave-7nm7"
server_key: "fa2cb832a9b46cadaee6ab0bd13aade1"
server_key_iv: "742ea8a56a61eddb661551237698fc43"
server_key_version: "encryptionkey@0"
I20260812 03:13:21.114951 22064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "3f70e7a14b1f4043ab810eee6b012765"
format_stamp: "Formatted at 2026-08-12 03:13:21 on dist-test-slave-7nm7"
server_key: "fa2cb832a9b46cadaee6ab0bd13aade1"
server_key_iv: "742ea8a56a61eddb661551237698fc43"
server_key_version: "encryptionkey@0"
I20260812 03:13:21.119005 22064 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 03:13:21.121491 22089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:21.122946 22064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:13:21.123136 22064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "3f70e7a14b1f4043ab810eee6b012765"
format_stamp: "Formatted at 2026-08-12 03:13:21 on dist-test-slave-7nm7"
server_key: "fa2cb832a9b46cadaee6ab0bd13aade1"
server_key_iv: "742ea8a56a61eddb661551237698fc43"
server_key_version: "encryptionkey@0"
I20260812 03:13:21.123262 22064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:21.149073 22064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:21.152330 22064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:21.152568 22064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:21.153244 22064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:21.154160 22064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:21.154239 22064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:21.154348 22064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:21.154408 22064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:21.164072 22064 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:45679
I20260812 03:13:21.164100 22292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:45679 every 8 connection(s)
I20260812 03:13:21.165040 22064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:13:21.168839 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 22064
I20260812 03:13:21.168970 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:13:21.169350 17253 external_mini_cluster.cc:1428] Setting key d0069218839e468784cc8121fb1087cb
Aug 12 03:13:21 dist-test-slave-7nm7 krb5kdc[21859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504401, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.217.66@KRBTEST.COM; defaulting to no policy
I20260812 03:13:21.180774 22019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:21.167115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:34343 (local address 127.16.217.126:37221)
0812 03:13:21.167274 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:21.167278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:21.168454 (+ 1176us) server_negotiation.cc:407] Connection header received
0812 03:13:21.169745 (+ 1291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:21.169748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:21.169831 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:21.169926 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:21.171355 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:21.172183 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:21.173497 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:21.173742 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:21.173860 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:21.176724 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:21.176729 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:21.176732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:21.176766 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:21.178615 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:21.179151 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:21.179155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:21.179157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:21.179243 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:21.179611 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:21.179615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:21.179617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:21.179818 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:21.179936 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:21.180458 (+ 522us) server_negotiation.cc:299] Negotiation successful
0812 03:13:21.180599 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:13:21.181731 22300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:21.167460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:37221 (local address 127.16.217.65:34343)
0812 03:13:21.168303 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:21.168348 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:21.169436 (+ 1088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:21.170157 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:21.170164 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:21.170588 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:21.171199 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:21.171215 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:21.172358 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:21.172361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:21.173354 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:21.173364 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:21.173904 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:21.173908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:21.174079 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:21.174727 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:21.174752 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:21.176548 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:21.178740 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:21.178747 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:21.178763 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:21.179038 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:21.179344 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:21.179348 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:21.179350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:21.179499 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:21.179935 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:21.179944 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:21.180212 (+ 268us) client_negotiation.cc:769] Sending connection context
0812 03:13:21.180416 (+ 204us) client_negotiation.cc:240] Negotiation successful
0812 03:13:21.180770 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":122,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.66@KRBTEST.COM" created.
I20260812 03:13:21.183344 22295 heartbeater.cc:347] Connected to a master server at 127.16.217.126:37221
I20260812 03:13:21.183643 22295 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:21.184238 22295 heartbeater.cc:511] Master 127.16.217.126:37221 requested a full tablet report, sending...
I20260812 03:13:21.186163 21947 ts_manager.cc:194] Registered new tserver with Master: 3f70e7a14b1f4043ab810eee6b012765 (127.16.217.65:45679)
I20260812 03:13:21.187474 21947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:34343
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
I20260812 03:13:21.202183 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:37221
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:34787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:21.321578 22305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:21.321923 22305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:21.322069 22305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:21.326217 22305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:21.326299 22305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:21.326416 22305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:13:21.331854 22305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:34787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:37221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:21.333019 22305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:21.334080 22305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:21.344759 22315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:21.344846 22305 server_base.cc:1034] running on GCE node
W20260812 03:13:21.344828 22321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:21.344760 22318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:21.345481 22305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:21.346101 22305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:21.347316 22305 hybrid_clock.cc:648] HybridClock initialized: now 1786504401347300 us; error 28 us; skew 500 ppm
Aug 12 03:13:21 dist-test-slave-7nm7 krb5kdc[21859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504401, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:21.350205 22305 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:13:21.351166 22305 webserver.cc:460] Webserver started at http://127.16.217.66:33571/ using document root <none> and password file <none>
I20260812 03:13:21.351694 22305 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:21.351754 22305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:21.351918 22305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:21.353904 22305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "dddcff79cef8454c99f133d989449ed2"
format_stamp: "Formatted at 2026-08-12 03:13:21 on dist-test-slave-7nm7"
server_key: "5cb9ca91d8af36ac3a91231cad34e4c5"
server_key_iv: "1c255d7ae588cb9fe66aed7c1469087b"
server_key_version: "encryptionkey@0"
I20260812 03:13:21.354496 22305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "dddcff79cef8454c99f133d989449ed2"
format_stamp: "Formatted at 2026-08-12 03:13:21 on dist-test-slave-7nm7"
server_key: "5cb9ca91d8af36ac3a91231cad34e4c5"
server_key_iv: "1c255d7ae588cb9fe66aed7c1469087b"
server_key_version: "encryptionkey@0"
I20260812 03:13:21.358155 22305 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:13:21.360664 22334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:21.362150 22305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:21.362440 22305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "dddcff79cef8454c99f133d989449ed2"
format_stamp: "Formatted at 2026-08-12 03:13:21 on dist-test-slave-7nm7"
server_key: "5cb9ca91d8af36ac3a91231cad34e4c5"
server_key_iv: "1c255d7ae588cb9fe66aed7c1469087b"
server_key_version: "encryptionkey@0"
I20260812 03:13:21.362579 22305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:21.388531 22305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:21.391841 22305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:21.392016 22305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:21.392617 22305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:21.393456 22305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:21.393508 22305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:21.393553 22305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:21.393586 22305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:21.403446 22305 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:42653
I20260812 03:13:21.403467 22548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:42653 every 8 connection(s)
I20260812 03:13:21.404383 22305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:13:21.410701 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 22305
I20260812 03:13:21.410830 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:13:21.411113 17253 external_mini_cluster.cc:1428] Setting key 7693e0bbf2851c8610bb0936871eceef
Aug 12 03:13:21 dist-test-slave-7nm7 krb5kdc[21859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504401, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:21.419643 22019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:21.406314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:53655 (local address 127.16.217.126:37221)
0812 03:13:21.406458 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:21.406462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:21.407498 (+ 1036us) server_negotiation.cc:407] Connection header received
0812 03:13:21.408397 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:21.408400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:21.408453 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:21.408540 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:21.409681 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:21.410355 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:21.411715 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:21.412007 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:21.412139 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:21.415365 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:21.415370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:21.415375 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:21.415411 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:21.417298 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:21.417834 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:21.417840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:21.417845 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:21.417957 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:21.418475 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:21.418480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:21.418482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:21.418690 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:21.418808 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:21.419336 (+ 528us) server_negotiation.cc:299] Negotiation successful
0812 03:13:21.419468 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:13:21.420459 22552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:21.406719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:37221 (local address 127.16.217.66:53655)
0812 03:13:21.407370 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:21.407409 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:21.408183 (+ 774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:21.408710 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:21.408717 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:21.409058 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:21.409519 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:21.409532 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:21.410488 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:21.410491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:21.411583 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:21.411596 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:21.412288 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:21.412291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:21.412533 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:21.413244 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:21.413270 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:21.415168 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:21.417427 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:21.417433 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:21.417455 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:21.417724 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:21.418113 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:21.418116 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:21.418117 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:21.418362 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:21.418808 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:21.418816 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:21.419067 (+ 251us) client_negotiation.cc:769] Sending connection context
0812 03:13:21.419275 (+ 208us) client_negotiation.cc:240] Negotiation successful
0812 03:13:21.419609 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":111,"threads_started":1}
I20260812 03:13:21.421664 22549 heartbeater.cc:347] Connected to a master server at 127.16.217.126:37221
I20260812 03:13:21.421954 22549 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.16.217.67@KRBTEST.COM; defaulting to no policy
I20260812 03:13:21.422626 22549 heartbeater.cc:511] Master 127.16.217.126:37221 requested a full tablet report, sending...
I20260812 03:13:21.423826 21947 ts_manager.cc:194] Registered new tserver with Master: dddcff79cef8454c99f133d989449ed2 (127.16.217.66:42653)
I20260812 03:13:21.424387 21947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:53655
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
I20260812 03:13:21.441992 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:37221
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:34787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:21.563345 22558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:21.563642 22558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:21.563688 22558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:21.567869 22558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:21.567953 22558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:21.568025 22558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:13:21.573863 22558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:34787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:37221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:21.575109 22558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:21.576267 22558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:21.587011 22571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:21.587034 22569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:21.587034 22568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:21.587442 22558 server_base.cc:1034] running on GCE node
I20260812 03:13:21.588057 22558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:21.588804 22558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:21.590003 22558 hybrid_clock.cc:648] HybridClock initialized: now 1786504401589963 us; error 60 us; skew 500 ppm
Aug 12 03:13:21 dist-test-slave-7nm7 krb5kdc[21859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504401, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:21.593089 22558 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:13:21.594128 22558 webserver.cc:460] Webserver started at http://127.16.217.67:46505/ using document root <none> and password file <none>
I20260812 03:13:21.594790 22558 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:21.594861 22558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:21.595088 22558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:21.597566 22558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "459fe93eef2b47b1a3ebdb41d856df64"
format_stamp: "Formatted at 2026-08-12 03:13:21 on dist-test-slave-7nm7"
server_key: "abc82f77c539cc7a7203e96c58ca2b17"
server_key_iv: "698fecad10eccfd7b5fa543f217f51d9"
server_key_version: "encryptionkey@0"
I20260812 03:13:21.598456 22558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "459fe93eef2b47b1a3ebdb41d856df64"
format_stamp: "Formatted at 2026-08-12 03:13:21 on dist-test-slave-7nm7"
server_key: "abc82f77c539cc7a7203e96c58ca2b17"
server_key_iv: "698fecad10eccfd7b5fa543f217f51d9"
server_key_version: "encryptionkey@0"
I20260812 03:13:21.603448 22558 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260812 03:13:21.606544 22580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:21.608261 22558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:21.608430 22558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "459fe93eef2b47b1a3ebdb41d856df64"
format_stamp: "Formatted at 2026-08-12 03:13:21 on dist-test-slave-7nm7"
server_key: "abc82f77c539cc7a7203e96c58ca2b17"
server_key_iv: "698fecad10eccfd7b5fa543f217f51d9"
server_key_version: "encryptionkey@0"
I20260812 03:13:21.608559 22558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:21.631026 22558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:21.635602 22558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:21.635856 22558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:21.636543 22558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:21.637485 22558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:21.637559 22558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:21.637635 22558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:21.637683 22558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:21.648562 22558 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:36115
I20260812 03:13:21.648607 22779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:36115 every 8 connection(s)
I20260812 03:13:21.649574 22558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:13:21.651309 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 22558
I20260812 03:13:21.651414 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:13:21.651732 17253 external_mini_cluster.cc:1428] Setting key 81e2055def13e6505829c34672e0013d
Aug 12 03:13:21 dist-test-slave-7nm7 krb5kdc[21859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504401, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:21.665225 22019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:21.651842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:51141 (local address 127.16.217.126:37221)
0812 03:13:21.652007 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:21.652012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:21.653255 (+ 1243us) server_negotiation.cc:407] Connection header received
0812 03:13:21.654173 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:21.654175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:21.654222 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:21.654324 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:21.655546 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:21.656097 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:21.657258 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:21.657473 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:21.657555 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:21.660244 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:21.660248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:21.660251 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:21.660432 (+ 181us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:21.662425 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:21.662980 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:21.662983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:21.662985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:21.663099 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:21.663578 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:21.663584 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:21.663585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:21.664054 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:21.664158 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:21.664819 (+ 661us) server_negotiation.cc:299] Negotiation successful
0812 03:13:21.664955 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 03:13:21.666608 22783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:21.652318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:37221 (local address 127.16.217.67:51141)
0812 03:13:21.653100 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:21.653154 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:21.653958 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:21.654545 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:21.654551 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:21.654931 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:21.655427 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:21.655441 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:21.656229 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:21.656232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:21.657132 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:21.657143 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:21.657616 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:21.657618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:21.657785 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:21.658482 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:21.658499 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:21.660056 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:21.662569 (+ 2513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:21.662574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:21.662589 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:21.662870 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:21.663211 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:21.663213 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:21.663214 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:21.663375 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:21.664198 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:21.664206 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:21.664535 (+ 329us) client_negotiation.cc:769] Sending connection context
0812 03:13:21.664763 (+ 228us) client_negotiation.cc:240] Negotiation successful
0812 03:13:21.665261 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":147,"threads_started":1}
I20260812 03:13:21.667994 22780 heartbeater.cc:347] Connected to a master server at 127.16.217.126:37221
I20260812 03:13:21.668254 22780 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:21.669585 22780 heartbeater.cc:511] Master 127.16.217.126:37221 requested a full tablet report, sending...
I20260812 03:13:21.671096 21947 ts_manager.cc:194] Registered new tserver with Master: 459fe93eef2b47b1a3ebdb41d856df64 (127.16.217.67:36115)
I20260812 03:13:21.671927 21947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:51141
I20260812 03:13:21.678479 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 03:13:21.695434 22019 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:21.693691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50140 (local address 127.16.217.126:37221)
0812 03:13:21.693841 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:21.693845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:21.693880 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:13:21.694020 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:21.694023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:21.694089 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:21.694184 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:21.695311 (+ 1127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50140: BlockingRecv error: recv got EOF from 127.0.0.1:50140 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:13:21.696540 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 22064
I20260812 03:13:21.720445 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 22305
I20260812 03:13:21.736650 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 22558
I20260812 03:13:21.752127 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 21887
2026-08-12T03:13:21Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2589 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:13:21 dist-test-slave-7nm7 krb5kdc[22799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:13:21 dist-test-slave-7nm7 krb5kdc[22799](info): set up 2 sockets
Aug 12 03:13:21 dist-test-slave-7nm7 krb5kdc[22799](info): commencing operation
krb5kdc: starting...
W20260812 03:13:23.099087 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:13:23 dist-test-slave-7nm7 krb5kdc[22799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:13:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:13:23.246639 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:46485
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:37943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:46485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:23.360359 22819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:23.360680 22819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:23.360723 22819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:23.364907 22819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:23.364988 22819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:23.365010 22819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:23.365027 22819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:23.365099 22819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:13:23.370888 22819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:46485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:46485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:23.372037 22819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:23.373104 22819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:23.382109 22832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:23.382117 22827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:23.382414 22819 server_base.cc:1034] running on GCE node
W20260812 03:13:23.382129 22829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:23.383189 22819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:23.384330 22819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:23.385522 22819 hybrid_clock.cc:648] HybridClock initialized: now 1786504403385489 us; error 45 us; skew 500 ppm
Aug 12 03:13:23 dist-test-slave-7nm7 krb5kdc[22799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504403, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:23.388489 22819 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:13:23.389480 22819 webserver.cc:460] Webserver started at http://127.16.217.126:36747/ using document root <none> and password file <none>
I20260812 03:13:23.390043 22819 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:23.390111 22819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:23.390376 22819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:23.392459 22819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "6bdf677845594083a17c4ebaf9602d67"
format_stamp: "Formatted at 2026-08-12 03:13:23 on dist-test-slave-7nm7"
server_key: "80dd65f602fa01a70664b881659c8643"
server_key_iv: "afde83c037e5c6ce244df56f73c807c9"
server_key_version: "encryptionkey@0"
I20260812 03:13:23.393067 22819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "6bdf677845594083a17c4ebaf9602d67"
format_stamp: "Formatted at 2026-08-12 03:13:23 on dist-test-slave-7nm7"
server_key: "80dd65f602fa01a70664b881659c8643"
server_key_iv: "afde83c037e5c6ce244df56f73c807c9"
server_key_version: "encryptionkey@0"
I20260812 03:13:23.397001 22819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:13:23.399575 22844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:23.400801 22819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:13:23.400908 22819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "6bdf677845594083a17c4ebaf9602d67"
format_stamp: "Formatted at 2026-08-12 03:13:23 on dist-test-slave-7nm7"
server_key: "80dd65f602fa01a70664b881659c8643"
server_key_iv: "afde83c037e5c6ce244df56f73c807c9"
server_key_version: "encryptionkey@0"
I20260812 03:13:23.401046 22819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:23.441715 22819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:23.445135 22819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:23.445371 22819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:23.452915 22819 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:46485
I20260812 03:13:23.453248 22941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:46485 every 8 connection(s)
I20260812 03:13:23.453972 22819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:23.457001 22944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:23.462679 22944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67: Bootstrap starting.
I20260812 03:13:23.464074 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 22819
I20260812 03:13:23.464219 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:13:23.464488 17253 external_mini_cluster.cc:1428] Setting key aaf74fdc28d02b8d2c4e92ab4fb6ac69
I20260812 03:13:23.465413 22944 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:23.466367 22944 log.cc:824] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:23.468641 22944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67: No bootstrap required, opened a new log
I20260812 03:13:23.473013 22944 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdf677845594083a17c4ebaf9602d67" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46485 } }
I20260812 03:13:23.473250 22944 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:23.473315 22944 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bdf677845594083a17c4ebaf9602d67, State: Initialized, Role: FOLLOWER
I20260812 03:13:23.473806 22944 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdf677845594083a17c4ebaf9602d67" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46485 } }
Aug 12 03:13:23 dist-test-slave-7nm7 krb5kdc[22799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:23.473975 22944 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:23.474090 22944 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:23.474205 22944 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:23.475275 22944 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdf677845594083a17c4ebaf9602d67" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46485 } }
I20260812 03:13:23.475643 22944 leader_election.cc:302] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bdf677845594083a17c4ebaf9602d67; no voters:
I20260812 03:13:23.476053 22944 leader_election.cc:288] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:13:23.476140 22949 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:23.476369 22949 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67 [term 1 LEADER]: Becoming Leader. State: Replica: 6bdf677845594083a17c4ebaf9602d67, State: Running, Role: LEADER
I20260812 03:13:23.476745 22949 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdf677845594083a17c4ebaf9602d67" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46485 } }
I20260812 03:13:23.477425 22944 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:23.477996 22949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bdf677845594083a17c4ebaf9602d67. Latest consensus state: current_term: 1 leader_uuid: "6bdf677845594083a17c4ebaf9602d67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdf677845594083a17c4ebaf9602d67" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46485 } } }
I20260812 03:13:23.478221 22949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:23.478452 22951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bdf677845594083a17c4ebaf9602d67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdf677845594083a17c4ebaf9602d67" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46485 } } }
I20260812 03:13:23.478560 22951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:23.478737 22964 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:23.480365 22947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:23.465692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41450 (local address 127.16.217.126:46485)
0812 03:13:23.466253 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:23.466277 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:23.466322 (+ 45us) server_negotiation.cc:407] Connection header received
0812 03:13:23.467101 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:23.467152 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:23.467553 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:23.467907 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:23.468587 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:23.470066 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:23.471031 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:23.471761 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:23.471984 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:23.474685 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:23.474691 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:23.474714 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:23.474743 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:23.476806 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:23.477296 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:23.477300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:23.477305 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:23.477459 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:23.477786 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:23.477790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:23.477792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:23.478404 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:23.478604 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:23.479061 (+ 457us) server_negotiation.cc:299] Negotiation successful
0812 03:13:23.479389 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":162,"threads_started":1}
I20260812 03:13:23.482642 22964 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:23.483763 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:13:23.488755 22964 catalog_manager.cc:1347] Generated new cluster ID: 5374a46303c946329a27211a7ee17222
I20260812 03:13:23.488821 22964 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:23.503014 22947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:23.498148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41456 (local address 127.16.217.126:46485)
0812 03:13:23.498303 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:23.498309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:23.498566 (+ 257us) server_negotiation.cc:407] Connection header received
0812 03:13:23.498737 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:23.498746 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:23.498792 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:23.498880 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:23.499394 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:23.499907 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:23.500753 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:23.500988 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:23.501103 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:23.501984 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:23.501988 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:23.501991 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:13:23.502525 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:23.502554 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:13:23.502571 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:23.502681 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:23.502796 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 03:13:23.502860 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:13:23.510700 22964 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:13:23.511981 22964 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:23.519802 22964 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bdf677845594083a17c4ebaf9602d67: Generated new TSK 0
I20260812 03:13:23.520501 22964 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:13:23.531592 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 22819
2026-08-12T03:13:23Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1771 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:13:23 dist-test-slave-7nm7 krb5kdc[23007](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:13:23 dist-test-slave-7nm7 krb5kdc[23007](info): set up 2 sockets
Aug 12 03:13:23 dist-test-slave-7nm7 krb5kdc[23007](info): commencing operation
krb5kdc: starting...
W20260812 03:13:24.848462 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:13:24 dist-test-slave-7nm7 krb5kdc[23007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:13:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:13:24.981846 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:45969
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:45969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:25.101323 23024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:25.101660 23024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:25.101747 23024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:25.105834 23024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:25.105938 23024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:25.105979 23024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:25.106043 23024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:25.106087 23024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:13:25.112154 23024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:45969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:45969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:25.113415 23024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:25.114634 23024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:25.124182 23037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:25.124325 23032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:25.124190 23040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:25.124464 23024 server_base.cc:1034] running on GCE node
I20260812 03:13:25.125201 23024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:25.126417 23024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:25.127645 23024 hybrid_clock.cc:648] HybridClock initialized: now 1786504405127629 us; error 41 us; skew 500 ppm
Aug 12 03:13:25 dist-test-slave-7nm7 krb5kdc[23007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504405, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:25.135814 23024 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:13:25.136842 23024 webserver.cc:460] Webserver started at http://127.16.217.126:37695/ using document root <none> and password file <none>
I20260812 03:13:25.137496 23024 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:25.137607 23024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:25.137859 23024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:25.139986 23024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "f37a457b14764a7eac561c7ff651d2cd"
format_stamp: "Formatted at 2026-08-12 03:13:25 on dist-test-slave-7nm7"
server_key: "480dfb787d0b89b14a361faccafef856"
server_key_iv: "75dad6b78cb649e39e860c13f8ec7848"
server_key_version: "encryptionkey@0"
I20260812 03:13:25.140569 23024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "f37a457b14764a7eac561c7ff651d2cd"
format_stamp: "Formatted at 2026-08-12 03:13:25 on dist-test-slave-7nm7"
server_key: "480dfb787d0b89b14a361faccafef856"
server_key_iv: "75dad6b78cb649e39e860c13f8ec7848"
server_key_version: "encryptionkey@0"
I20260812 03:13:25.144336 23024 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:13:25.146839 23053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:25.148052 23024 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:13:25.148203 23024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "f37a457b14764a7eac561c7ff651d2cd"
format_stamp: "Formatted at 2026-08-12 03:13:25 on dist-test-slave-7nm7"
server_key: "480dfb787d0b89b14a361faccafef856"
server_key_iv: "75dad6b78cb649e39e860c13f8ec7848"
server_key_version: "encryptionkey@0"
I20260812 03:13:25.148308 23024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:25.182212 23024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:25.185649 23024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:25.185904 23024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:25.193318 23024 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:45969
I20260812 03:13:25.193614 23163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:45969 every 8 connection(s)
I20260812 03:13:25.194456 23024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:25.197309 23165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:25.199604 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 23024
I20260812 03:13:25.199738 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:13:25.200037 17253 external_mini_cluster.cc:1428] Setting key 6227d1525721a39b601c3586e0d4d27c
I20260812 03:13:25.203763 23165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd: Bootstrap starting.
I20260812 03:13:25.206007 23165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:25.206885 23165 log.cc:824] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:25.208913 23165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd: No bootstrap required, opened a new log
Aug 12 03:13:25 dist-test-slave-7nm7 krb5kdc[23007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:25.213537 23165 raft_consensus.cc:348] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37a457b14764a7eac561c7ff651d2cd" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 45969 } }
I20260812 03:13:25.213915 23165 raft_consensus.cc:374] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:25.213984 23165 raft_consensus.cc:729] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f37a457b14764a7eac561c7ff651d2cd, State: Initialized, Role: FOLLOWER
I20260812 03:13:25.214604 23165 consensus_queue.cc:260] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37a457b14764a7eac561c7ff651d2cd" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 45969 } }
I20260812 03:13:25.214766 23165 raft_consensus.cc:388] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:25.214856 23165 raft_consensus.cc:482] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:25.215011 23165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:25.215039 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:25.201557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54494 (local address 127.16.217.126:45969)
0812 03:13:25.202004 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:25.202019 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:25.202053 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:13:25.202906 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:25.202925 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:25.203261 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:25.203599 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:25.204405 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:25.205400 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:25.206347 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:25.206801 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:25.206968 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:25.209663 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:25.209668 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:25.209681 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:25.209709 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:25.211472 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:25.211996 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:25.212001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:25.212006 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:25.212235 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:25.212573 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:25.212575 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:25.212576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:25.213000 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:25.213175 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:25.213647 (+ 472us) server_negotiation.cc:299] Negotiation successful
0812 03:13:25.214027 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":138,"threads_started":1}
I20260812 03:13:25.216049 23165 raft_consensus.cc:504] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37a457b14764a7eac561c7ff651d2cd" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 45969 } }
I20260812 03:13:25.216405 23165 leader_election.cc:302] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f37a457b14764a7eac561c7ff651d2cd; no voters:
I20260812 03:13:25.216799 23165 leader_election.cc:288] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:13:25.216872 23172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:25.217099 23172 raft_consensus.cc:686] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd [term 1 LEADER]: Becoming Leader. State: Replica: f37a457b14764a7eac561c7ff651d2cd, State: Running, Role: LEADER
I20260812 03:13:25.217449 23172 consensus_queue.cc:237] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37a457b14764a7eac561c7ff651d2cd" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 45969 } }
I20260812 03:13:25.218144 23165 sys_catalog.cc:564] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:25.218710 23172 sys_catalog.cc:455] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f37a457b14764a7eac561c7ff651d2cd. Latest consensus state: current_term: 1 leader_uuid: "f37a457b14764a7eac561c7ff651d2cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37a457b14764a7eac561c7ff651d2cd" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 45969 } } }
I20260812 03:13:25.218890 23172 sys_catalog.cc:458] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:25.219167 23178 sys_catalog.cc:455] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f37a457b14764a7eac561c7ff651d2cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37a457b14764a7eac561c7ff651d2cd" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 45969 } } }
I20260812 03:13:25.219352 23188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:25.219354 23178 sys_catalog.cc:458] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:25.222352 23188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:25.228370 23188 catalog_manager.cc:1347] Generated new cluster ID: fb97ef35a86d4a40a1d4c2e7ec28dc18
I20260812 03:13:25.228437 23188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:25.252380 23188 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:13:25.253746 23188 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:25.271984 23188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f37a457b14764a7eac561c7ff651d2cd: Generated new TSK 0
I20260812 03:13:25.272876 23188 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
I20260812 03:13:25.305460 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:45969
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:25.423796 23212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:25.424057 23212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:25.424100 23212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:25.428205 23212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:25.428265 23212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:25.428323 23212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:13:25.433653 23212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:45969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:25.434830 23212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:25.435869 23212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:25.448330 23220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:25.449630 23223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:25.452523 23219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:25.453210 23212 server_base.cc:1034] running on GCE node
I20260812 03:13:25.453832 23212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:25.454761 23212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:25.455971 23212 hybrid_clock.cc:648] HybridClock initialized: now 1786504405455952 us; error 39 us; skew 500 ppm
Aug 12 03:13:25 dist-test-slave-7nm7 krb5kdc[23007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504405, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:25.460013 23212 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:13:25.461263 23212 webserver.cc:460] Webserver started at http://127.16.217.65:39677/ using document root <none> and password file <none>
I20260812 03:13:25.462302 23212 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:25.462402 23212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:25.462692 23212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:25.465969 23212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "e5412d10391647ba8a8ac30bf83e3a50"
format_stamp: "Formatted at 2026-08-12 03:13:25 on dist-test-slave-7nm7"
server_key: "e5c9432e3adb3968e22846682673f012"
server_key_iv: "4ddd03ae6390ee858d806604bf01843e"
server_key_version: "encryptionkey@0"
I20260812 03:13:25.466797 23212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "e5412d10391647ba8a8ac30bf83e3a50"
format_stamp: "Formatted at 2026-08-12 03:13:25 on dist-test-slave-7nm7"
server_key: "e5c9432e3adb3968e22846682673f012"
server_key_iv: "4ddd03ae6390ee858d806604bf01843e"
server_key_version: "encryptionkey@0"
I20260812 03:13:25.473768 23212 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260812 03:13:25.476845 23232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:25.478391 23212 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:25.478498 23212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "e5412d10391647ba8a8ac30bf83e3a50"
format_stamp: "Formatted at 2026-08-12 03:13:25 on dist-test-slave-7nm7"
server_key: "e5c9432e3adb3968e22846682673f012"
server_key_iv: "4ddd03ae6390ee858d806604bf01843e"
server_key_version: "encryptionkey@0"
I20260812 03:13:25.478595 23212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:25.489312 23212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:25.492379 23212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:25.492542 23212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:25.493157 23212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:25.493986 23212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:25.494035 23212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:25.494076 23212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:25.494098 23212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:25.504215 23212 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:39489
I20260812 03:13:25.504302 23443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:39489 every 8 connection(s)
I20260812 03:13:25.505192 23212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:13:25.513567 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 23212
I20260812 03:13:25.513717 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:13:25.514006 17253 external_mini_cluster.cc:1428] Setting key cfe3690410f11342c8026c420c59da38
Aug 12 03:13:25 dist-test-slave-7nm7 krb5kdc[23007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504405, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:25.519763 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:25.507129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:36049 (local address 127.16.217.126:45969)
0812 03:13:25.507281 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:25.507292 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:25.508125 (+ 833us) server_negotiation.cc:407] Connection header received
0812 03:13:25.509495 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:25.509498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:25.509561 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:25.509672 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:25.510915 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:25.511469 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:25.512619 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:25.512853 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:25.512969 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:25.515869 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:25.515873 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:25.515875 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:25.515899 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:25.517386 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:25.517932 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:25.517942 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:25.517944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:25.518030 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:25.518482 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:25.518486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:25.518488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:25.518753 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:25.518866 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:25.519421 (+ 555us) server_negotiation.cc:299] Negotiation successful
0812 03:13:25.519550 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:13:25.520671 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:25.507483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:45969 (local address 127.16.217.65:36049)
0812 03:13:25.507966 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:25.508009 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:25.509280 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:25.509859 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:25.509865 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:25.510221 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:25.510760 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:25.510773 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:25.511608 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:25.511612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:25.512481 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:25.512491 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:25.513011 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:25.513016 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:25.513187 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:25.513936 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:25.513953 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:25.515695 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:25.517523 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:25.517529 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:25.517545 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:25.517817 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:25.518131 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:25.518134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:25.518136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:25.518380 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:25.518865 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:25.518873 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:25.519129 (+ 256us) client_negotiation.cc:769] Sending connection context
0812 03:13:25.519326 (+ 197us) client_negotiation.cc:240] Negotiation successful
0812 03:13:25.519671 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":117,"threads_started":1}
I20260812 03:13:25.522059 23445 heartbeater.cc:347] Connected to a master server at 127.16.217.126:45969
I20260812 03:13:25.522333 23445 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:25.522846 23445 heartbeater.cc:511] Master 127.16.217.126:45969 requested a full tablet report, sending...
I20260812 03:13:25.524438 23088 ts_manager.cc:194] Registered new tserver with Master: e5412d10391647ba8a8ac30bf83e3a50 (127.16.217.65:39489)
WARNING: no policy specified for kudu/127.16.217.66@KRBTEST.COM; defaulting to no policy
I20260812 03:13:25.525624 23088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:36049
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
I20260812 03:13:25.544826 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:45969
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:25.656059 23453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:25.656322 23453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:25.656390 23453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:25.660398 23453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:25.660480 23453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:25.660576 23453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:13:25.665729 23453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:45969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:25.666790 23453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:25.667779 23453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:25.677172 23464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:25.677328 23453 server_base.cc:1034] running on GCE node
W20260812 03:13:25.677239 23462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:25.677170 23461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:25.677762 23453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:25.678438 23453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:25.679621 23453 hybrid_clock.cc:648] HybridClock initialized: now 1786504405679587 us; error 45 us; skew 500 ppm
Aug 12 03:13:25 dist-test-slave-7nm7 krb5kdc[23007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504405, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:25.682487 23453 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:13:25.683307 23453 webserver.cc:460] Webserver started at http://127.16.217.66:42079/ using document root <none> and password file <none>
I20260812 03:13:25.683796 23453 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:25.683854 23453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:25.684011 23453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:25.685869 23453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "4f46b860d7d7449198a1d87a834abc47"
format_stamp: "Formatted at 2026-08-12 03:13:25 on dist-test-slave-7nm7"
server_key: "47a585e92ddbd7c29579f82fcbe13ce2"
server_key_iv: "8c3329582bb2cadc13a0a909d397713f"
server_key_version: "encryptionkey@0"
I20260812 03:13:25.686440 23453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "4f46b860d7d7449198a1d87a834abc47"
format_stamp: "Formatted at 2026-08-12 03:13:25 on dist-test-slave-7nm7"
server_key: "47a585e92ddbd7c29579f82fcbe13ce2"
server_key_iv: "8c3329582bb2cadc13a0a909d397713f"
server_key_version: "encryptionkey@0"
I20260812 03:13:25.689877 23453 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:13:25.692288 23476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:25.693415 23453 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:13:25.693508 23453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "4f46b860d7d7449198a1d87a834abc47"
format_stamp: "Formatted at 2026-08-12 03:13:25 on dist-test-slave-7nm7"
server_key: "47a585e92ddbd7c29579f82fcbe13ce2"
server_key_iv: "8c3329582bb2cadc13a0a909d397713f"
server_key_version: "encryptionkey@0"
I20260812 03:13:25.693585 23453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:25.724016 23453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:25.727357 23453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:25.727514 23453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:25.728055 23453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:25.728852 23453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:25.728901 23453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:25.728941 23453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:25.728965 23453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:25.738119 23453 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:34197
I20260812 03:13:25.738148 23673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:34197 every 8 connection(s)
I20260812 03:13:25.739053 23453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:13:25.742035 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 23453
I20260812 03:13:25.742182 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:13:25.742530 17253 external_mini_cluster.cc:1428] Setting key 6d8fafc307f1fde8bf53d205e1cb16c8
Aug 12 03:13:25 dist-test-slave-7nm7 krb5kdc[23007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504405, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.217.67@KRBTEST.COM; defaulting to no policy
I20260812 03:13:25.753723 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:25.740837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:48773 (local address 127.16.217.126:45969)
0812 03:13:25.741008 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:25.741013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:25.742055 (+ 1042us) server_negotiation.cc:407] Connection header received
0812 03:13:25.743363 (+ 1308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:25.743367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:25.743415 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:25.743509 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:25.744763 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:25.745592 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:25.746827 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:25.747241 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:25.747372 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:25.749789 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:25.749792 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:25.749794 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:25.749819 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:25.751364 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:25.752028 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:25.752032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:25.752035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:25.752139 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:25.752535 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:25.752537 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:25.752538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:25.752714 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:25.752790 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:25.753434 (+ 644us) server_negotiation.cc:299] Negotiation successful
0812 03:13:25.753586 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 03:13:25.754678 23676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:25.741228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:45969 (local address 127.16.217.66:48773)
0812 03:13:25.741672 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:25.741714 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:25.743162 (+ 1448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:25.743656 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:25.743662 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:25.744123 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:25.744589 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:25.744602 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:25.745728 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:25.745731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:25.746693 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:25.746702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:25.747388 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:25.747390 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:25.747518 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:25.748102 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:25.748118 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:25.749614 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:25.751488 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:25.751497 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:25.751513 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:25.751886 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:25.752248 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:25.752252 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:25.752255 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:25.752433 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:25.752832 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:25.752841 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:25.753165 (+ 324us) client_negotiation.cc:769] Sending connection context
0812 03:13:25.753387 (+ 222us) client_negotiation.cc:240] Negotiation successful
0812 03:13:25.753736 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":102,"threads_started":1}
I20260812 03:13:25.755882 23674 heartbeater.cc:347] Connected to a master server at 127.16.217.126:45969
I20260812 03:13:25.756139 23674 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.67@KRBTEST.COM" created.
I20260812 03:13:25.756724 23674 heartbeater.cc:511] Master 127.16.217.126:45969 requested a full tablet report, sending...
I20260812 03:13:25.757823 23088 ts_manager.cc:194] Registered new tserver with Master: 4f46b860d7d7449198a1d87a834abc47 (127.16.217.66:34197)
I20260812 03:13:25.758545 23088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:48773
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
I20260812 03:13:25.773109 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:45969
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:25.891108 23681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:25.891448 23681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:25.891525 23681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:25.895982 23681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:25.896095 23681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:25.896204 23681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:13:25.901758 23681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:45969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:25.902989 23681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:25.904273 23681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:25.914930 23697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:25.914943 23695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:25.914808 23690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:25.915217 23681 server_base.cc:1034] running on GCE node
I20260812 03:13:25.915759 23681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:25.916437 23681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:25.917649 23681 hybrid_clock.cc:648] HybridClock initialized: now 1786504405917626 us; error 33 us; skew 500 ppm
Aug 12 03:13:25 dist-test-slave-7nm7 krb5kdc[23007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504405, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:25.921005 23681 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:13:25.921933 23681 webserver.cc:460] Webserver started at http://127.16.217.67:39945/ using document root <none> and password file <none>
I20260812 03:13:25.922573 23681 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:25.922665 23681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:25.922912 23681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:25.925127 23681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "009dd718115944a29b850395d85ac139"
format_stamp: "Formatted at 2026-08-12 03:13:25 on dist-test-slave-7nm7"
server_key: "d5e0d2a8015078cb5f689d4084d5ebf5"
server_key_iv: "ecc43b9170f04c270034bebb65c24c8f"
server_key_version: "encryptionkey@0"
I20260812 03:13:25.925781 23681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "009dd718115944a29b850395d85ac139"
format_stamp: "Formatted at 2026-08-12 03:13:25 on dist-test-slave-7nm7"
server_key: "d5e0d2a8015078cb5f689d4084d5ebf5"
server_key_iv: "ecc43b9170f04c270034bebb65c24c8f"
server_key_version: "encryptionkey@0"
I20260812 03:13:25.929667 23681 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260812 03:13:25.932176 23704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:25.933494 23681 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:13:25.933637 23681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "009dd718115944a29b850395d85ac139"
format_stamp: "Formatted at 2026-08-12 03:13:25 on dist-test-slave-7nm7"
server_key: "d5e0d2a8015078cb5f689d4084d5ebf5"
server_key_iv: "ecc43b9170f04c270034bebb65c24c8f"
server_key_version: "encryptionkey@0"
I20260812 03:13:25.933760 23681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:25.951536 23681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:25.955087 23681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:25.955309 23681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:25.955945 23681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:25.956808 23681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:25.956892 23681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:25.956990 23681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:25.957043 23681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:25.966161 23681 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:35877
I20260812 03:13:25.966197 23902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:35877 every 8 connection(s)
I20260812 03:13:25.967161 23681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:13:25.970449 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 23681
I20260812 03:13:25.970587 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:13:25.970934 17253 external_mini_cluster.cc:1428] Setting key ffcaf8822b7a52e17542b76aaeffc1df
Aug 12 03:13:25 dist-test-slave-7nm7 krb5kdc[23007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504405, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:25.982157 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:25.969783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:44703 (local address 127.16.217.126:45969)
0812 03:13:25.969944 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:25.969949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:25.971044 (+ 1095us) server_negotiation.cc:407] Connection header received
0812 03:13:25.972826 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:25.972829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:25.972885 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:25.972995 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:25.974355 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:25.974949 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:25.976094 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:25.976345 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:25.976428 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:25.978946 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:25.978949 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:25.978951 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:25.978985 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:25.980311 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:25.980775 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:25.980777 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:25.980779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:25.980851 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:25.981143 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:25.981145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:25.981146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:25.981310 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:25.981384 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:25.981931 (+ 547us) server_negotiation.cc:299] Negotiation successful
0812 03:13:25.982034 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:13:25.983035 23907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:25.970282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:45969 (local address 127.16.217.67:44703)
0812 03:13:25.970874 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:25.970932 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:25.972620 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:25.973200 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:25.973209 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:25.973575 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:25.974061 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:25.974074 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:25.975078 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:25.975081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:25.975953 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:25.975962 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:25.976499 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:25.976509 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:25.976649 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:25.977222 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:25.977239 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:25.978787 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:25.980421 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:25.980426 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:25.980438 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:25.980685 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:25.980933 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:25.980935 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:25.980936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:25.981061 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:25.981406 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:25.981411 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:25.981701 (+ 290us) client_negotiation.cc:769] Sending connection context
0812 03:13:25.981863 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 03:13:25.982103 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":138,"threads_started":1}
I20260812 03:13:25.984194 23904 heartbeater.cc:347] Connected to a master server at 127.16.217.126:45969
I20260812 03:13:25.984465 23904 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:25.985016 23904 heartbeater.cc:511] Master 127.16.217.126:45969 requested a full tablet report, sending...
I20260812 03:13:25.985991 23087 ts_manager.cc:194] Registered new tserver with Master: 009dd718115944a29b850395d85ac139 (127.16.217.67:35877)
I20260812 03:13:25.986608 23087 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:44703
I20260812 03:13:25.996659 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:13:26 dist-test-slave-7nm7 krb5kdc[23007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504406, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 03:13:26 dist-test-slave-7nm7 krb5kdc[23007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504406, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:26.024012 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:26.015068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54506 (local address 127.16.217.126:45969)
0812 03:13:26.015245 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:26.015250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:26.015406 (+ 156us) server_negotiation.cc:407] Connection header received
0812 03:13:26.015582 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:26.015586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:26.015640 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:26.015735 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:26.016455 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:26.017001 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:26.017859 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:26.018096 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:26.018212 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:26.020831 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:26.020835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:26.020838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:26.020873 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:26.022215 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:26.022689 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:26.022694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:26.022703 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:26.022789 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:26.023074 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:26.023078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:26.023079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:26.023312 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:26.023457 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:26.023711 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 03:13:26.023834 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260812 03:13:26.024473 23087 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54506
I20260812 03:13:26.025352 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 23212
I20260812 03:13:26.064939 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 23453
I20260812 03:13:26.075600 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 23681
I20260812 03:13:26.091733 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 23024
2026-08-12T03:13:26Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2560 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:13:26 dist-test-slave-7nm7 krb5kdc[23928](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:13:26 dist-test-slave-7nm7 krb5kdc[23928](info): set up 2 sockets
Aug 12 03:13:26 dist-test-slave-7nm7 krb5kdc[23928](info): commencing operation
krb5kdc: starting...
W20260812 03:13:27.416656 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:13:27 dist-test-slave-7nm7 krb5kdc[23928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:13:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:13:27.546298 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:37757
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:36291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:37757
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:27.658646 23957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:27.658952 23957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:27.658995 23957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:27.663020 23957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:27.663082 23957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:27.663101 23957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:27.663118 23957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:27.663134 23957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:13:27.668922 23957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:36291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:37757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:37757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:27.670161 23957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:27.671288 23957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:27.681592 23970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:27.681680 23966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:27.681592 23973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:27.681999 23957 server_base.cc:1034] running on GCE node
I20260812 03:13:27.682773 23957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:27.684005 23957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:27.685214 23957 hybrid_clock.cc:648] HybridClock initialized: now 1786504407685177 us; error 55 us; skew 500 ppm
Aug 12 03:13:27 dist-test-slave-7nm7 krb5kdc[23928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504407, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:27.689206 23957 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:13:27.690331 23957 webserver.cc:460] Webserver started at http://127.16.217.126:41727/ using document root <none> and password file <none>
I20260812 03:13:27.690943 23957 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:27.691051 23957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:27.691304 23957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:27.693478 23957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "1db3fb63fd31412fb7bc5940be13f9d8"
format_stamp: "Formatted at 2026-08-12 03:13:27 on dist-test-slave-7nm7"
server_key: "9ea29ba37833cc392afdd9212697b7fc"
server_key_iv: "eccdadb796efc4be13c54335fe3de480"
server_key_version: "encryptionkey@0"
I20260812 03:13:27.694082 23957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "1db3fb63fd31412fb7bc5940be13f9d8"
format_stamp: "Formatted at 2026-08-12 03:13:27 on dist-test-slave-7nm7"
server_key: "9ea29ba37833cc392afdd9212697b7fc"
server_key_iv: "eccdadb796efc4be13c54335fe3de480"
server_key_version: "encryptionkey@0"
I20260812 03:13:27.698493 23957 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:13:27.701064 23982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:27.702600 23957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:27.702750 23957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "1db3fb63fd31412fb7bc5940be13f9d8"
format_stamp: "Formatted at 2026-08-12 03:13:27 on dist-test-slave-7nm7"
server_key: "9ea29ba37833cc392afdd9212697b7fc"
server_key_iv: "eccdadb796efc4be13c54335fe3de480"
server_key_version: "encryptionkey@0"
I20260812 03:13:27.702872 23957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:27.719012 23957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:27.722705 23957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:27.722937 23957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:27.731518 23957 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:37757
I20260812 03:13:27.731561 24074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:37757 every 8 connection(s)
I20260812 03:13:27.732574 23957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:27.732982 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 23957
I20260812 03:13:27.733152 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:13:27.733467 17253 external_mini_cluster.cc:1428] Setting key b488b1895219e61300d7f30b0cbd9dd6
I20260812 03:13:27.735826 24077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:27.743252 24077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8: Bootstrap starting.
Aug 12 03:13:27 dist-test-slave-7nm7 krb5kdc[23928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:27.745855 24077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:27.746683 24077 log.cc:824] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:27.748754 24077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8: No bootstrap required, opened a new log
I20260812 03:13:27.749505 24082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:27.735394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49220 (local address 127.16.217.126:37757)
0812 03:13:27.735843 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:27.735858 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:27.735924 (+ 66us) server_negotiation.cc:407] Connection header received
0812 03:13:27.736591 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:27.736607 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:27.736924 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:27.737199 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:27.738193 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:27.739389 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:27.740710 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:27.741184 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:27.741323 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:27.744546 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:27.744552 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:27.744565 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:27.744602 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:27.746591 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:27.747109 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:27.747114 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:27.747120 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:27.747244 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:27.747539 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:27.747542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:27.747544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:27.747949 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:27.748151 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:27.748458 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 03:13:27.748747 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":146,"threads_started":1}
I20260812 03:13:27.752869 24077 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db3fb63fd31412fb7bc5940be13f9d8" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 37757 } }
I20260812 03:13:27.753145 24077 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:27.753201 24077 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1db3fb63fd31412fb7bc5940be13f9d8, State: Initialized, Role: FOLLOWER
I20260812 03:13:27.753639 24077 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db3fb63fd31412fb7bc5940be13f9d8" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 37757 } }
I20260812 03:13:27.753829 24077 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:27.753929 24077 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:27.754051 24077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:27.755079 24077 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db3fb63fd31412fb7bc5940be13f9d8" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 37757 } }
I20260812 03:13:27.755434 24077 leader_election.cc:302] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1db3fb63fd31412fb7bc5940be13f9d8; no voters:
I20260812 03:13:27.755774 24077 leader_election.cc:288] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:13:27.755934 24086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:27.756224 24086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8 [term 1 LEADER]: Becoming Leader. State: Replica: 1db3fb63fd31412fb7bc5940be13f9d8, State: Running, Role: LEADER
I20260812 03:13:27.756546 24086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db3fb63fd31412fb7bc5940be13f9d8" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 37757 } }
I20260812 03:13:27.757179 24077 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:27.758296 24091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1db3fb63fd31412fb7bc5940be13f9d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db3fb63fd31412fb7bc5940be13f9d8" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 37757 } } }
I20260812 03:13:27.758440 24091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:27.758455 24092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1db3fb63fd31412fb7bc5940be13f9d8. Latest consensus state: current_term: 1 leader_uuid: "1db3fb63fd31412fb7bc5940be13f9d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db3fb63fd31412fb7bc5940be13f9d8" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 37757 } } }
I20260812 03:13:27.758590 24092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:27.759899 24102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:27.763267 24102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:27.769132 24102 catalog_manager.cc:1347] Generated new cluster ID: af9654f7b80146cbb157e97e91fd0bc7
I20260812 03:13:27.769210 24102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:27.799549 24102 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:13:27.800587 24102 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:27.809432 24102 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1db3fb63fd31412fb7bc5940be13f9d8: Generated new TSK 0
I20260812 03:13:27.810141 24102 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
I20260812 03:13:27.847414 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:37757
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:36291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:27.964614 24127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:27.964919 24127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:27.964965 24127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:27.969008 24127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:27.969069 24127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:27.969129 24127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:13:27.975111 24127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:36291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:37757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:27.976305 24127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:27.977610 24127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:27.988103 24132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:27.988123 24135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:27.988220 24127 server_base.cc:1034] running on GCE node
W20260812 03:13:27.988129 24133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:27.988749 24127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:27.989418 24127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:27.990586 24127 hybrid_clock.cc:648] HybridClock initialized: now 1786504407990552 us; error 45 us; skew 500 ppm
Aug 12 03:13:27 dist-test-slave-7nm7 krb5kdc[23928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504407, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:27.993484 24127 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:13:27.994448 24127 webserver.cc:460] Webserver started at http://127.16.217.65:40295/ using document root <none> and password file <none>
I20260812 03:13:27.994998 24127 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:27.995059 24127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:27.995235 24127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:27.997260 24127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "2b42407a52774d8699682c9cabd162a5"
format_stamp: "Formatted at 2026-08-12 03:13:27 on dist-test-slave-7nm7"
server_key: "fd3fbf2c5db02fe962404d517f65581c"
server_key_iv: "27302c0f76732d86b28a16f339c03aad"
server_key_version: "encryptionkey@0"
I20260812 03:13:27.997782 24127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "2b42407a52774d8699682c9cabd162a5"
format_stamp: "Formatted at 2026-08-12 03:13:27 on dist-test-slave-7nm7"
server_key: "fd3fbf2c5db02fe962404d517f65581c"
server_key_iv: "27302c0f76732d86b28a16f339c03aad"
server_key_version: "encryptionkey@0"
I20260812 03:13:28.001626 24127 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:13:28.004341 24146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:28.005733 24127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 03:13:28.005872 24127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "2b42407a52774d8699682c9cabd162a5"
format_stamp: "Formatted at 2026-08-12 03:13:27 on dist-test-slave-7nm7"
server_key: "fd3fbf2c5db02fe962404d517f65581c"
server_key_iv: "27302c0f76732d86b28a16f339c03aad"
server_key_version: "encryptionkey@0"
I20260812 03:13:28.005995 24127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:28.042666 24127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:28.045912 24127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:28.046120 24127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:28.046823 24127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:28.047709 24127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:28.047784 24127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:28.047860 24127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:28.047911 24127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:28.059638 24127 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:35245
I20260812 03:13:28.059662 24343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:35245 every 8 connection(s)
I20260812 03:13:28.060621 24127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:13:28.066118 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 24127
I20260812 03:13:28.066311 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:13:28.066753 17253 external_mini_cluster.cc:1428] Setting key d7159506779a05c3486a677b554f7236
Aug 12 03:13:28 dist-test-slave-7nm7 krb5kdc[23928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504407, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:28.078332 24082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:28.062835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:41657 (local address 127.16.217.126:37757)
0812 03:13:28.062999 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:28.063004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:28.064149 (+ 1145us) server_negotiation.cc:407] Connection header received
0812 03:13:28.066177 (+ 2028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:28.066181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:28.066291 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:28.066390 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:28.067813 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:28.068552 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:28.069766 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:28.070213 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:28.070372 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:28.073767 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:28.073771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:28.073774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:28.073802 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:28.075632 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:28.076266 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:28.076270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:28.076272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:28.076357 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:28.076761 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:28.076765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:28.076767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:28.077011 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:28.077138 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:28.077969 (+ 831us) server_negotiation.cc:299] Negotiation successful
0812 03:13:28.078125 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
WARNING: no policy specified for kudu/127.16.217.66@KRBTEST.COM; defaulting to no policy
I20260812 03:13:28.079291 24350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:28.063314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:37757 (local address 127.16.217.65:41657)
0812 03:13:28.063964 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:28.064028 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:28.065860 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:28.066595 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:28.066602 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:28.066977 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:28.067655 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:28.067676 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:28.068695 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:28.068698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:28.069626 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:28.069637 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:28.070423 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:28.070430 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:28.070678 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:28.071571 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:28.071602 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:28.073520 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:28.075788 (+ 2268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:28.075796 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:28.075816 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:28.076124 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:28.076470 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:28.076474 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:28.076476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:28.076646 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:28.077152 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:28.077161 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:28.077663 (+ 502us) client_negotiation.cc:769] Sending connection context
0812 03:13:28.077901 (+ 238us) client_negotiation.cc:240] Negotiation successful
0812 03:13:28.078333 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":119,"threads_started":1}
I20260812 03:13:28.080647 24346 heartbeater.cc:347] Connected to a master server at 127.16.217.126:37757
I20260812 03:13:28.080942 24346 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:28.081666 24346 heartbeater.cc:511] Master 127.16.217.126:37757 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.66@KRBTEST.COM" created.
I20260812 03:13:28.083472 24007 ts_manager.cc:194] Registered new tserver with Master: 2b42407a52774d8699682c9cabd162a5 (127.16.217.65:35245)
I20260812 03:13:28.085054 24007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:41657
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
I20260812 03:13:28.102806 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:37757
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:36291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:28.228163 24353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:28.228439 24353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:28.228484 24353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:28.232918 24353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:28.232980 24353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:28.233078 24353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:13:28.238359 24353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:36291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:37757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:28.239391 24353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:28.240447 24353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:13:28.250639 24353 server_base.cc:1034] running on GCE node
W20260812 03:13:28.250821 24363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:28.250864 24360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:28.250844 24361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:28.251466 24353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:28.252025 24353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:28.253201 24353 hybrid_clock.cc:648] HybridClock initialized: now 1786504408253182 us; error 32 us; skew 500 ppm
Aug 12 03:13:28 dist-test-slave-7nm7 krb5kdc[23928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504408, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:28.256098 24353 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:13:28.256986 24353 webserver.cc:460] Webserver started at http://127.16.217.66:38999/ using document root <none> and password file <none>
I20260812 03:13:28.257551 24353 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:28.257644 24353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:28.257905 24353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:28.260142 24353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "269e0f398c6445ea870ea0c01d38c5ff"
format_stamp: "Formatted at 2026-08-12 03:13:28 on dist-test-slave-7nm7"
server_key: "e8d35709235bee693829ed136b9538c7"
server_key_iv: "0d185aadf791da85003c4883e09ab77a"
server_key_version: "encryptionkey@0"
I20260812 03:13:28.260715 24353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "269e0f398c6445ea870ea0c01d38c5ff"
format_stamp: "Formatted at 2026-08-12 03:13:28 on dist-test-slave-7nm7"
server_key: "e8d35709235bee693829ed136b9538c7"
server_key_iv: "0d185aadf791da85003c4883e09ab77a"
server_key_version: "encryptionkey@0"
I20260812 03:13:28.264331 24353 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:13:28.266742 24376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:28.267870 24353 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:13:28.268008 24353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "269e0f398c6445ea870ea0c01d38c5ff"
format_stamp: "Formatted at 2026-08-12 03:13:28 on dist-test-slave-7nm7"
server_key: "e8d35709235bee693829ed136b9538c7"
server_key_iv: "0d185aadf791da85003c4883e09ab77a"
server_key_version: "encryptionkey@0"
I20260812 03:13:28.268119 24353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:28.280695 24353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:28.283679 24353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:28.283897 24353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:28.284504 24353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:28.285557 24353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:28.285642 24353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:28.285740 24353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:28.285794 24353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:28.295244 24353 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:35349
I20260812 03:13:28.295272 24587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:35349 every 8 connection(s)
I20260812 03:13:28.296123 24353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:13:28.302037 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 24353
I20260812 03:13:28.302162 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:13:28.302538 17253 external_mini_cluster.cc:1428] Setting key c2f97d230971c4431203c73941bf12ed
Aug 12 03:13:28 dist-test-slave-7nm7 krb5kdc[23928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504408, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:28.311139 24082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:28.297993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:57913 (local address 127.16.217.126:37757)
0812 03:13:28.298172 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:28.298177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:28.299004 (+ 827us) server_negotiation.cc:407] Connection header received
0812 03:13:28.300322 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:28.300325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:28.300382 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:28.300489 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:28.301574 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:28.302398 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:28.303737 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:28.304036 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:28.304171 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:28.306851 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:28.306855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:28.306857 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:28.306882 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:28.308510 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:28.309155 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:28.309160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:28.309162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:28.309264 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:28.309863 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:28.309867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:28.309869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:28.310068 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:28.310188 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:28.310815 (+ 627us) server_negotiation.cc:299] Negotiation successful
0812 03:13:28.310947 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:13:28.311973 24593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:28.298411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:37757 (local address 127.16.217.66:57913)
0812 03:13:28.298855 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:28.298895 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:28.300101 (+ 1206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:28.300635 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:28.300641 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:28.300973 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:28.301420 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:28.301433 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:28.302548 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:28.302551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:28.303607 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:28.303618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:28.304220 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:28.304223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:28.304360 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:28.305006 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:28.305023 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:28.306669 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:28.308662 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:28.308668 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:28.308683 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:28.309036 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:28.309373 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:28.309378 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:28.309380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:28.309741 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:28.310203 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:28.310212 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:28.310497 (+ 285us) client_negotiation.cc:769] Sending connection context
0812 03:13:28.310706 (+ 209us) client_negotiation.cc:240] Negotiation successful
0812 03:13:28.311049 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.16.217.67@KRBTEST.COM; defaulting to no policy
I20260812 03:13:28.313223 24588 heartbeater.cc:347] Connected to a master server at 127.16.217.126:37757
I20260812 03:13:28.313481 24588 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:28.313941 24588 heartbeater.cc:511] Master 127.16.217.126:37757 requested a full tablet report, sending...
I20260812 03:13:28.315061 24007 ts_manager.cc:194] Registered new tserver with Master: 269e0f398c6445ea870ea0c01d38c5ff (127.16.217.66:35349)
I20260812 03:13:28.315722 24007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:57913
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
I20260812 03:13:28.332930 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:37757
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:36291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:28.452981 24600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:28.453248 24600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:28.453293 24600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:28.457471 24600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:28.457530 24600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:28.457645 24600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:13:28.463104 24600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:36291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:37757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:28.464195 24600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:28.465520 24600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:28.475631 24609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:28.475641 24610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:28.475714 24600 server_base.cc:1034] running on GCE node
W20260812 03:13:28.475901 24612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:28.476464 24600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:28.477171 24600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:28.478339 24600 hybrid_clock.cc:648] HybridClock initialized: now 1786504408478312 us; error 60 us; skew 500 ppm
Aug 12 03:13:28 dist-test-slave-7nm7 krb5kdc[23928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504408, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:28.481184 24600 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:13:28.482095 24600 webserver.cc:460] Webserver started at http://127.16.217.67:35401/ using document root <none> and password file <none>
I20260812 03:13:28.482741 24600 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:28.482837 24600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:28.483086 24600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:28.485212 24600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "23e2e65a589944778eed72a42096dcf9"
format_stamp: "Formatted at 2026-08-12 03:13:28 on dist-test-slave-7nm7"
server_key: "608cbc5b61b1267c1bdc344a44d84f0f"
server_key_iv: "9e83482e1a02227a5f42b14005555bae"
server_key_version: "encryptionkey@0"
I20260812 03:13:28.485792 24600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "23e2e65a589944778eed72a42096dcf9"
format_stamp: "Formatted at 2026-08-12 03:13:28 on dist-test-slave-7nm7"
server_key: "608cbc5b61b1267c1bdc344a44d84f0f"
server_key_iv: "9e83482e1a02227a5f42b14005555bae"
server_key_version: "encryptionkey@0"
I20260812 03:13:28.489773 24600 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:13:28.492619 24622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:28.494086 24600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 03:13:28.494320 24600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "23e2e65a589944778eed72a42096dcf9"
format_stamp: "Formatted at 2026-08-12 03:13:28 on dist-test-slave-7nm7"
server_key: "608cbc5b61b1267c1bdc344a44d84f0f"
server_key_iv: "9e83482e1a02227a5f42b14005555bae"
server_key_version: "encryptionkey@0"
I20260812 03:13:28.494448 24600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:28.507822 24600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:28.512212 24600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:28.512445 24600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:28.513113 24600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:28.514034 24600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:28.514112 24600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:28.514184 24600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:28.514228 24600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:28.524382 24600 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:43891
I20260812 03:13:28.524406 24824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:43891 every 8 connection(s)
I20260812 03:13:28.525372 24600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:13:28.530911 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 24600
I20260812 03:13:28.531204 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:13:28.531575 17253 external_mini_cluster.cc:1428] Setting key 4aa696714b9b0c5631f61e606ef26525
Aug 12 03:13:28 dist-test-slave-7nm7 krb5kdc[23928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504408, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:28.540191 24082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:28.527454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:59845 (local address 127.16.217.126:37757)
0812 03:13:28.527635 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:28.527639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:28.528493 (+ 854us) server_negotiation.cc:407] Connection header received
0812 03:13:28.529917 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:28.529921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:28.529982 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:28.530103 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:28.531652 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:28.532304 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:28.533641 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:28.533914 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:28.534022 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:28.536488 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:28.536492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:28.536495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:28.536529 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:28.537949 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:28.538584 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:28.538588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:28.538590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:28.538679 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:28.539033 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:28.539036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:28.539038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:28.539241 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:28.539353 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:28.539882 (+ 529us) server_negotiation.cc:299] Negotiation successful
0812 03:13:28.540005 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 03:13:28.541077 24828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:28.527831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:37757 (local address 127.16.217.67:59845)
0812 03:13:28.528333 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:28.528375 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:28.529702 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:28.530339 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:28.530349 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:28.530734 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:28.531473 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:28.531491 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:28.532433 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:28.532437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:28.533495 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:28.533505 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:28.534061 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:28.534065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:28.534275 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:28.534827 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:28.534845 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:28.536319 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:28.538102 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:28.538108 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:28.538125 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:28.538474 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:28.538777 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:28.538780 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:28.538782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:28.538926 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:28.539353 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:28.539362 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:28.539625 (+ 263us) client_negotiation.cc:769] Sending connection context
0812 03:13:28.539823 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 03:13:28.540156 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":129,"threads_started":1}
I20260812 03:13:28.542614 24825 heartbeater.cc:347] Connected to a master server at 127.16.217.126:37757
I20260812 03:13:28.542922 24825 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:28.543457 24825 heartbeater.cc:511] Master 127.16.217.126:37757 requested a full tablet report, sending...
I20260812 03:13:28.544835 24007 ts_manager.cc:194] Registered new tserver with Master: 23e2e65a589944778eed72a42096dcf9 (127.16.217.67:43891)
I20260812 03:13:28.545585 24007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:59845
I20260812 03:13:28.546085 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:13:28 dist-test-slave-7nm7 krb5kdc[23928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:13:28 dist-test-slave-7nm7 krb5kdc[23928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.65@KRBTEST.COM
I20260812 03:13:28.575331 24835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:28.564302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39676 (local address 127.16.217.65:35245)
0812 03:13:28.564641 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:28.564656 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:28.564681 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:13:28.564748 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:28.564753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:28.564944 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:28.565065 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:28.565695 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:28.566591 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:28.567572 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:28.567954 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:28.568066 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:28.570692 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:28.570702 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:28.570717 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:28.570743 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:28.572973 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:28.573712 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:28.573718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:28.573720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:28.573825 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:28.574213 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:28.574216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:28.574218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:28.574572 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:28.574752 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:28.575017 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 03:13:28.575155 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":132,"threads_started":1}
Aug 12 03:13:28 dist-test-slave-7nm7 krb5kdc[23928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:28.587131 24082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:28.578601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49226 (local address 127.16.217.126:37757)
0812 03:13:28.578760 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:28.578763 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:28.578779 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:13:28.578873 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:28.578876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:28.578919 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:28.578987 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:28.579586 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:28.580225 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:28.580994 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:28.581220 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:28.581306 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:28.583748 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:28.583751 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:28.583753 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:28.583778 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:28.585297 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:28.585757 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:28.585760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:28.585761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:28.585854 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:28.586213 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:28.586216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:28.586217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:28.586533 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:28.586651 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:28.586889 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 03:13:28.587007 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260812 03:13:28.587595 24007 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49226
I20260812 03:13:28.588279 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 24127
I20260812 03:13:28.601204 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 24353
I20260812 03:13:28.621343 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 24600
I20260812 03:13:28.634872 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 23957
2026-08-12T03:13:28Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2541 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:13:28 dist-test-slave-7nm7 krb5kdc[24841](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:13:28 dist-test-slave-7nm7 krb5kdc[24841](info): set up 2 sockets
Aug 12 03:13:28 dist-test-slave-7nm7 krb5kdc[24841](info): commencing operation
krb5kdc: starting...
W20260812 03:13:29.942871 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:13:30 dist-test-slave-7nm7 krb5kdc[24841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:13:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:13:30.068914 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:38227
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:33883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:38227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:30.191829 24861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:30.192123 24861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:30.192215 24861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:30.196259 24861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:30.196345 24861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:30.196384 24861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:30.196424 24861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:30.196457 24861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:13:30.201835 24861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:33883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:38227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:38227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:30.203001 24861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:30.204038 24861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:30.212981 24867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:30.213006 24869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:30.212987 24866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:30.213274 24861 server_base.cc:1034] running on GCE node
I20260812 03:13:30.213972 24861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:30.215147 24861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:30.216341 24861 hybrid_clock.cc:648] HybridClock initialized: now 1786504410216322 us; error 33 us; skew 500 ppm
Aug 12 03:13:30 dist-test-slave-7nm7 krb5kdc[24841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504410, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:30.222577 24861 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:13:30.223393 24861 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:30.223461 24861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:30.223687 24861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:30.225812 24861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "23da92fedf944408b1753d2ab719ab1f"
format_stamp: "Formatted at 2026-08-12 03:13:30 on dist-test-slave-7nm7"
server_key: "b4d36f517b31a96cf3a27ba4bb67c944"
server_key_iv: "d09cb3bab9ac2970eee59b84318ff986"
server_key_version: "encryptionkey@0"
I20260812 03:13:30.226488 24861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "23da92fedf944408b1753d2ab719ab1f"
format_stamp: "Formatted at 2026-08-12 03:13:30 on dist-test-slave-7nm7"
server_key: "b4d36f517b31a96cf3a27ba4bb67c944"
server_key_iv: "d09cb3bab9ac2970eee59b84318ff986"
server_key_version: "encryptionkey@0"
I20260812 03:13:30.230325 24861 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:13:30.232784 24876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:30.234104 24861 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 03:13:30.234287 24861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "23da92fedf944408b1753d2ab719ab1f"
format_stamp: "Formatted at 2026-08-12 03:13:30 on dist-test-slave-7nm7"
server_key: "b4d36f517b31a96cf3a27ba4bb67c944"
server_key_iv: "d09cb3bab9ac2970eee59b84318ff986"
server_key_version: "encryptionkey@0"
I20260812 03:13:30.234409 24861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:30.248492 24861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:30.251585 24861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:30.251798 24861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:30.260092 24861 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:38227
I20260812 03:13:30.260392 24980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:38227 every 8 connection(s)
I20260812 03:13:30.260810 24861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:30.263569 24981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:30.267762 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 24861
I20260812 03:13:30.267930 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:13:30.268196 17253 external_mini_cluster.cc:1428] Setting key 9ef9457b511b8346d988518e914de36e
I20260812 03:13:30.269675 24981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f: Bootstrap starting.
I20260812 03:13:30.272002 24981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:30.272768 24981 log.cc:824] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:30.274753 24981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f: No bootstrap required, opened a new log
Aug 12 03:13:30 dist-test-slave-7nm7 krb5kdc[24841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:30.278532 24981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23da92fedf944408b1753d2ab719ab1f" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 38227 } }
I20260812 03:13:30.278789 24981 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:30.278874 24981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23da92fedf944408b1753d2ab719ab1f, State: Initialized, Role: FOLLOWER
I20260812 03:13:30.279372 24981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23da92fedf944408b1753d2ab719ab1f" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 38227 } }
I20260812 03:13:30.279529 24981 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:30.279620 24981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:30.279738 24981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:30.280895 24981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23da92fedf944408b1753d2ab719ab1f" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 38227 } }
I20260812 03:13:30.281293 24981 leader_election.cc:302] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23da92fedf944408b1753d2ab719ab1f; no voters:
I20260812 03:13:30.281642 24981 leader_election.cc:288] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:13:30.281759 24991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:30.281973 24991 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f [term 1 LEADER]: Becoming Leader. State: Replica: 23da92fedf944408b1753d2ab719ab1f, State: Running, Role: LEADER
I20260812 03:13:30.282743 24986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:30.269583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60638 (local address 127.16.217.126:38227)
0812 03:13:30.270021 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:30.270037 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:30.270072 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:13:30.270774 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:30.270789 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:30.271113 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:30.271468 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:30.272333 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:30.273372 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:30.274254 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:30.274721 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:30.274898 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:30.277336 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:30.277342 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:30.277355 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:30.277382 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:30.279220 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:30.279712 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:30.279719 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:30.279725 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:30.279869 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:30.280241 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:30.280243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:30.280245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:30.280626 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:30.280778 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:30.281197 (+ 419us) server_negotiation.cc:299] Negotiation successful
0812 03:13:30.281525 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":127,"threads_started":1}
I20260812 03:13:30.282925 24981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:30.283056 24991 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23da92fedf944408b1753d2ab719ab1f" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 38227 } }
I20260812 03:13:30.284698 24994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23da92fedf944408b1753d2ab719ab1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23da92fedf944408b1753d2ab719ab1f" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 38227 } } }
I20260812 03:13:30.284833 24994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:30.285259 24997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23da92fedf944408b1753d2ab719ab1f. Latest consensus state: current_term: 1 leader_uuid: "23da92fedf944408b1753d2ab719ab1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23da92fedf944408b1753d2ab719ab1f" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 38227 } } }
I20260812 03:13:30.285667 24997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f [sys.catalog]: This master's current role is: LEADER
W20260812 03:13:30.288646 25028 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:13:30.288738 25028 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:13:30.288805 25026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:30.289472 25026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:30.294669 25026 catalog_manager.cc:1347] Generated new cluster ID: b598706eddf14e4396545dfee8a8b14a
I20260812 03:13:30.294756 25026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:30.323546 25026 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:13:30.324854 25026 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:30.334550 25026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f: Generated new TSK 0
I20260812 03:13:30.335343 25026 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
I20260812 03:13:30.367676 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:38227
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:33883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:30.484776 25036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:30.485031 25036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:30.485077 25036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:30.489032 25036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:30.489094 25036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:30.489161 25036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:13:30.494812 25036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:33883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:38227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:30.496001 25036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:30.497067 25036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:30.507066 25043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:30.507066 25046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:30.507110 25044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:30.507387 25036 server_base.cc:1034] running on GCE node
I20260812 03:13:30.507887 25036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:30.508637 25036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:30.509826 25036 hybrid_clock.cc:648] HybridClock initialized: now 1786504410509805 us; error 34 us; skew 500 ppm
Aug 12 03:13:30 dist-test-slave-7nm7 krb5kdc[24841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504410, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:30.512970 25036 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:13:30.513799 25036 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:30.513868 25036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:30.514088 25036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:30.516251 25036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "23d2dd3324d74e58b0c561a22ee68407"
format_stamp: "Formatted at 2026-08-12 03:13:30 on dist-test-slave-7nm7"
server_key: "7a9b6da0c9869dc717b769c52afb7cec"
server_key_iv: "2e44dcdcb8e696e75488e0b162ce845c"
server_key_version: "encryptionkey@0"
I20260812 03:13:30.516829 25036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "23d2dd3324d74e58b0c561a22ee68407"
format_stamp: "Formatted at 2026-08-12 03:13:30 on dist-test-slave-7nm7"
server_key: "7a9b6da0c9869dc717b769c52afb7cec"
server_key_iv: "2e44dcdcb8e696e75488e0b162ce845c"
server_key_version: "encryptionkey@0"
I20260812 03:13:30.520759 25036 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:13:30.523336 25057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:30.524534 25036 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:13:30.524698 25036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "23d2dd3324d74e58b0c561a22ee68407"
format_stamp: "Formatted at 2026-08-12 03:13:30 on dist-test-slave-7nm7"
server_key: "7a9b6da0c9869dc717b769c52afb7cec"
server_key_iv: "2e44dcdcb8e696e75488e0b162ce845c"
server_key_version: "encryptionkey@0"
I20260812 03:13:30.524828 25036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:30.541724 25036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:30.545282 25036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:30.545526 25036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:30.546175 25036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:30.547142 25036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:30.547216 25036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:30.547295 25036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:30.547348 25036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:30.557392 25036 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:42197
I20260812 03:13:30.557410 25261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:42197 every 8 connection(s)
I20260812 03:13:30.558081 25036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:13:30.564872 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 25036
I20260812 03:13:30.565047 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:13:30.565353 17253 external_mini_cluster.cc:1428] Setting key 50b1478ae3acb7ed3d9d43ef00d156c6
Aug 12 03:13:30 dist-test-slave-7nm7 krb5kdc[24841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504410, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:30.574534 24986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:30.559996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:43557 (local address 127.16.217.126:38227)
0812 03:13:30.560144 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:30.560148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:30.561014 (+ 866us) server_negotiation.cc:407] Connection header received
0812 03:13:30.562440 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:30.562444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:30.562508 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:30.562627 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:30.563833 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:30.564385 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:30.565702 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:30.566025 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:30.566156 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:30.569861 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:30.569868 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:30.569872 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:30.569908 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:30.571880 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:30.572576 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:30.572591 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:30.572594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:30.572690 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:30.573047 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:30.573054 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:30.573058 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:30.573341 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:30.573463 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:30.573996 (+ 533us) server_negotiation.cc:299] Negotiation successful
0812 03:13:30.574272 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:13:30.575256 25269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:30.560345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:38227 (local address 127.16.217.65:43557)
0812 03:13:30.560851 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:30.560896 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:30.562167 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:30.562805 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:30.562812 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:30.563183 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:30.563676 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:30.563690 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:30.564529 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:30.564532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:30.565570 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:30.565587 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:30.566658 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:30.566661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:30.566809 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:30.567449 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:30.567466 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:30.569655 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:30.572035 (+ 2380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:30.572046 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:30.572065 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:30.572456 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:30.572799 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:30.572801 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:30.572803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:30.572941 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:30.573464 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:30.573470 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:30.573738 (+ 268us) client_negotiation.cc:769] Sending connection context
0812 03:13:30.573905 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 03:13:30.574216 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":128,"threads_started":1}
WARNING: no policy specified for kudu/127.16.217.66@KRBTEST.COM; defaulting to no policy
I20260812 03:13:30.576970 25262 heartbeater.cc:347] Connected to a master server at 127.16.217.126:38227
I20260812 03:13:30.577291 25262 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:30.577816 25262 heartbeater.cc:511] Master 127.16.217.126:38227 requested a full tablet report, sending...
I20260812 03:13:30.579576 24913 ts_manager.cc:194] Registered new tserver with Master: 23d2dd3324d74e58b0c561a22ee68407 (127.16.217.65:42197)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.66@KRBTEST.COM" created.
I20260812 03:13:30.581055 24913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:43557
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
I20260812 03:13:30.598141 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:38227
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:33883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:30.715502 25273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:30.715793 25273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:30.715871 25273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:30.719806 25273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:30.719901 25273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:30.719990 25273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:13:30.725090 25273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:33883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:38227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:30.726122 25273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:30.727154 25273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:30.736423 25287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:30.736438 25280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:30.736429 25281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:30.736847 25273 server_base.cc:1034] running on GCE node
I20260812 03:13:30.737264 25273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:30.737888 25273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:30.739049 25273 hybrid_clock.cc:648] HybridClock initialized: now 1786504410738971 us; error 89 us; skew 500 ppm
Aug 12 03:13:30 dist-test-slave-7nm7 krb5kdc[24841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504410, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:30.741757 25273 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:13:30.742687 25273 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:30.742784 25273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:30.743000 25273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:30.745033 25273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "bd56af40dad645adb2c51e124e676d17"
format_stamp: "Formatted at 2026-08-12 03:13:30 on dist-test-slave-7nm7"
server_key: "136f8035965c656d462290537761867e"
server_key_iv: "b6501adf75d821836e5e9fd812304eaf"
server_key_version: "encryptionkey@0"
I20260812 03:13:30.745608 25273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "bd56af40dad645adb2c51e124e676d17"
format_stamp: "Formatted at 2026-08-12 03:13:30 on dist-test-slave-7nm7"
server_key: "136f8035965c656d462290537761867e"
server_key_iv: "b6501adf75d821836e5e9fd812304eaf"
server_key_version: "encryptionkey@0"
I20260812 03:13:30.749430 25273 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:13:30.751966 25296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:30.753208 25273 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:13:30.753355 25273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "bd56af40dad645adb2c51e124e676d17"
format_stamp: "Formatted at 2026-08-12 03:13:30 on dist-test-slave-7nm7"
server_key: "136f8035965c656d462290537761867e"
server_key_iv: "b6501adf75d821836e5e9fd812304eaf"
server_key_version: "encryptionkey@0"
I20260812 03:13:30.753465 25273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:30.767460 25273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:30.770411 25273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:30.770603 25273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:30.771188 25273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:30.772020 25273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:30.772102 25273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:30.772176 25273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:30.772226 25273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:30.781410 25273 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:35647
I20260812 03:13:30.781410 25484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:35647 every 8 connection(s)
I20260812 03:13:30.782049 25273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:13:30.786346 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 25273
I20260812 03:13:30.786468 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:13:30.786792 17253 external_mini_cluster.cc:1428] Setting key 3945aa1fbc764f476c08ba795d4bac54
Aug 12 03:13:30 dist-test-slave-7nm7 krb5kdc[24841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504410, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:30.797246 24986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:30.784006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:47495 (local address 127.16.217.126:38227)
0812 03:13:30.784199 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:30.784203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:30.785012 (+ 809us) server_negotiation.cc:407] Connection header received
0812 03:13:30.786582 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:30.786586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:30.786645 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:30.786719 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:30.788146 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:30.788677 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:30.789826 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:30.790113 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:30.790280 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:30.793001 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:30.793007 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:30.793014 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:30.793050 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:30.794909 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:30.795480 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:30.795484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:30.795486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:30.795582 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:30.795933 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:30.795937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:30.795938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:30.796209 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:30.796328 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:30.796863 (+ 535us) server_negotiation.cc:299] Negotiation successful
0812 03:13:30.797015 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
WARNING: no policy specified for kudu/127.16.217.67@KRBTEST.COM; defaulting to no policy
I20260812 03:13:30.798071 25488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:30.784414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:38227 (local address 127.16.217.66:47495)
0812 03:13:30.784866 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:30.784908 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:30.786328 (+ 1420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:30.786900 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:30.786910 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:30.787355 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:30.787985 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:30.788006 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:30.788810 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:30.788814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:30.789681 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:30.789701 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:30.790318 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:30.790322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:30.790477 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:30.791102 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:30.791119 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:30.792815 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:30.795056 (+ 2241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:30.795067 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:30.795091 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:30.795367 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:30.795687 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:30.795689 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:30.795691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:30.795830 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:30.796346 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:30.796352 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:30.796604 (+ 252us) client_negotiation.cc:769] Sending connection context
0812 03:13:30.796770 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 03:13:30.797056 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":116,"threads_started":1}
I20260812 03:13:30.799439 25485 heartbeater.cc:347] Connected to a master server at 127.16.217.126:38227
I20260812 03:13:30.799702 25485 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:30.800230 25485 heartbeater.cc:511] Master 127.16.217.126:38227 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.67@KRBTEST.COM" created.
I20260812 03:13:30.801404 24913 ts_manager.cc:194] Registered new tserver with Master: bd56af40dad645adb2c51e124e676d17 (127.16.217.66:35647)
I20260812 03:13:30.801990 24913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:47495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
I20260812 03:13:30.817564 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:38227
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:33883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:30.933810 25492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:30.934051 25492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:30.934095 25492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:30.938156 25492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:30.938215 25492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:30.938289 25492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:13:30.944531 25492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:33883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:38227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:30.945613 25492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:30.946666 25492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:30.956882 25509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:30.956969 25512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:30.957167 25492 server_base.cc:1034] running on GCE node
W20260812 03:13:30.956894 25508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:30.957816 25492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:30.958578 25492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:30.959796 25492 hybrid_clock.cc:648] HybridClock initialized: now 1786504410959782 us; error 26 us; skew 500 ppm
Aug 12 03:13:30 dist-test-slave-7nm7 krb5kdc[24841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504410, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:30.962896 25492 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:13:30.963699 25492 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:30.963850 25492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:30.964115 25492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:30.966198 25492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "9d66658161d041af9b3dcfd548cb2e14"
format_stamp: "Formatted at 2026-08-12 03:13:30 on dist-test-slave-7nm7"
server_key: "0504dad5c6c9b5e6f959508993de7de2"
server_key_iv: "b302a3a2427a10132079ce84b96393be"
server_key_version: "encryptionkey@0"
I20260812 03:13:30.966810 25492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "9d66658161d041af9b3dcfd548cb2e14"
format_stamp: "Formatted at 2026-08-12 03:13:30 on dist-test-slave-7nm7"
server_key: "0504dad5c6c9b5e6f959508993de7de2"
server_key_iv: "b302a3a2427a10132079ce84b96393be"
server_key_version: "encryptionkey@0"
I20260812 03:13:30.970533 25492 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:30.972996 25522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:30.974412 25492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:30.974536 25492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "9d66658161d041af9b3dcfd548cb2e14"
format_stamp: "Formatted at 2026-08-12 03:13:30 on dist-test-slave-7nm7"
server_key: "0504dad5c6c9b5e6f959508993de7de2"
server_key_iv: "b302a3a2427a10132079ce84b96393be"
server_key_version: "encryptionkey@0"
I20260812 03:13:30.974645 25492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:31.000694 25492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:31.004211 25492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:31.004431 25492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:31.005079 25492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:31.005954 25492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:31.006027 25492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:31.006101 25492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:31.006152 25492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:31.016475 25492 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:37285
I20260812 03:13:31.016494 25736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:37285 every 8 connection(s)
I20260812 03:13:31.017307 25492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:13:31.025177 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 25492
I20260812 03:13:31.025321 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:13:31.025682 17253 external_mini_cluster.cc:1428] Setting key 2f2ef0ffece39fccd3737aa3b9f457c8
Aug 12 03:13:31 dist-test-slave-7nm7 krb5kdc[24841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504410, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:31.032691 24986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:31.019246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:60363 (local address 127.16.217.126:38227)
0812 03:13:31.019414 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:31.019420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:31.020337 (+ 917us) server_negotiation.cc:407] Connection header received
0812 03:13:31.021611 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:31.021614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:31.021677 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:31.021759 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:31.023114 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.023648 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.024789 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.025002 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.025085 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:31.028464 (+ 3379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:31.028469 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:31.028472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:31.028504 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:31.030417 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:31.031033 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:31.031037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:31.031038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:31.031151 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:31.031533 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:31.031537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:31.031538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:31.031756 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:31.031849 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:31.032434 (+ 585us) server_negotiation.cc:299] Negotiation successful
0812 03:13:31.032559 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 03:13:31.033540 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:31.019663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:38227 (local address 127.16.217.67:60363)
0812 03:13:31.020178 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:31.020223 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:31.021408 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:31.021927 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:31.021934 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:31.022351 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:31.022913 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:31.022936 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.023774 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.023776 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:31.024655 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:31.024664 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.025193 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.025201 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:31.025418 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:31.026114 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:31.026133 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:31.028232 (+ 2099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:31.030628 (+ 2396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:31.030634 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:31.030648 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:31.030901 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:31.031256 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:31.031258 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:31.031260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:31.031398 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:31.031910 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:31.031915 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:31.032172 (+ 257us) client_negotiation.cc:769] Sending connection context
0812 03:13:31.032353 (+ 181us) client_negotiation.cc:240] Negotiation successful
0812 03:13:31.032621 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":119,"threads_started":1}
I20260812 03:13:31.034828 25740 heartbeater.cc:347] Connected to a master server at 127.16.217.126:38227
I20260812 03:13:31.035063 25740 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:31.035507 25740 heartbeater.cc:511] Master 127.16.217.126:38227 requested a full tablet report, sending...
I20260812 03:13:31.036533 24913 ts_manager.cc:194] Registered new tserver with Master: 9d66658161d041af9b3dcfd548cb2e14 (127.16.217.67:37285)
I20260812 03:13:31.037163 24913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:60363
I20260812 03:13:31.039866 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:13:31.049605 24986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:31.041872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60640 (local address 127.16.217.126:38227)
0812 03:13:31.042028 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:31.042033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:31.042290 (+ 257us) server_negotiation.cc:407] Connection header received
0812 03:13:31.042410 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:31.042413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:31.042455 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:31.042523 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:31.043216 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.043716 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.044594 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.044817 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.044897 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:31.046218 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:31.046221 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:31.046223 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:31.046326 (+ 103us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:31.047776 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:31.048250 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:31.048253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:31.048255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:31.048333 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:31.048635 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:31.048638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:31.048639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:31.048866 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:31.048974 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:31.049245 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 03:13:31.049367 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:13:31.052627 24913 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:13:31.055791 24913 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:13:31.071308 25765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:31.065714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:35647 (local address 127.0.0.1:53984)
0812 03:13:31.065991 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:31.066003 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:31.066095 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:31.066916 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:31.066920 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:31.066936 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:31.067044 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:31.067052 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.068444 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.068448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:31.069718 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:31.069729 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.070872 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.070875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:31.070950 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:31.070986 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 03:13:31.071054 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:13:31.071148 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":107,"threads_started":1}
I20260812 03:13:31.071404 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:31.065126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54464 (local address 127.16.217.67:37285)
0812 03:13:31.065479 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:31.065492 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:31.065519 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:13:31.065753 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:31.065762 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:31.065997 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:31.066164 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:31.066740 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.067917 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.069457 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.070395 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.070505 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:31.070672 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:31.071140 (+ 468us) server_negotiation.cc:299] Negotiation successful
0812 03:13:31.071250 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":132,"threads_started":1}
I20260812 03:13:31.071735 25760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:31.064926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:37285 (local address 127.0.0.1:54464)
0812 03:13:31.065347 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:31.065402 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:31.065563 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:31.066426 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:31.066435 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:31.066463 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:31.066604 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:31.066614 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.068052 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.068055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:31.069308 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:31.069318 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.070733 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.070736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:31.070849 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:31.070892 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 03:13:31.071004 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 03:13:31.071598 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":492,"thread_start_us":125,"threads_started":1}
I20260812 03:13:31.071898 25769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:31.066070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:42197 (local address 127.0.0.1:39900)
0812 03:13:31.066341 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:31.066355 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:31.066440 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:31.067664 (+ 1224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:31.067667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:31.067688 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:31.067784 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:31.067790 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.069035 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.069039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:31.070188 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:31.070200 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.071465 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.071470 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:31.071577 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:31.071599 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:13:31.071665 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 03:13:31.071733 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":100,"threads_started":1}
I20260812 03:13:31.071758 25767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:31.065980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53984 (local address 127.16.217.66:35647)
0812 03:13:31.066483 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:31.066497 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:31.066519 (+ 22us) server_negotiation.cc:407] Connection header received
0812 03:13:31.066593 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:31.066596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:31.066767 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:31.066922 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:31.067321 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.068294 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.069863 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.070747 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.070864 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:31.071002 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:31.071101 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 03:13:31.071219 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":150,"threads_started":1}
I20260812 03:13:31.072363 25770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:31.066558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39900 (local address 127.16.217.65:42197)
0812 03:13:31.067247 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:31.067253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:31.067274 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:13:31.067344 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:31.067348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:31.067512 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:31.067657 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:31.067918 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.068881 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.070330 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.071136 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.071253 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:31.071385 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:31.072083 (+ 698us) server_negotiation.cc:299] Negotiation successful
0812 03:13:31.072192 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":114,"threads_started":1}
I20260812 03:13:31.075430 25617 tablet_service.cc:1467] Processing CreateTablet for tablet b2407511eb2b416a807c56217aa1df59 (DEFAULT_TABLE table=test-table [id=4013212299b341268f476d1184e4e740]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:31.075444 25378 tablet_service.cc:1467] Processing CreateTablet for tablet b2407511eb2b416a807c56217aa1df59 (DEFAULT_TABLE table=test-table [id=4013212299b341268f476d1184e4e740]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:31.076761 25378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2407511eb2b416a807c56217aa1df59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:31.077075 25617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2407511eb2b416a807c56217aa1df59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:31.080657 25138 tablet_service.cc:1467] Processing CreateTablet for tablet b2407511eb2b416a807c56217aa1df59 (DEFAULT_TABLE table=test-table [id=4013212299b341268f476d1184e4e740]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:31.081810 25138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2407511eb2b416a807c56217aa1df59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:31.085593 25771 tablet_bootstrap.cc:492] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14: Bootstrap starting.
I20260812 03:13:31.088599 25771 tablet_bootstrap.cc:654] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:31.090013 25771 log.cc:824] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:31.091598 25774 tablet_bootstrap.cc:492] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407: Bootstrap starting.
I20260812 03:13:31.094120 25771 tablet_bootstrap.cc:492] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14: No bootstrap required, opened a new log
I20260812 03:13:31.094364 25771 ts_tablet_manager.cc:1397] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 03:13:31.094864 25774 tablet_bootstrap.cc:654] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:31.095618 25774 log.cc:824] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:31.097031 25778 tablet_bootstrap.cc:492] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17: Bootstrap starting.
I20260812 03:13:31.097615 25774 tablet_bootstrap.cc:492] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407: No bootstrap required, opened a new log
I20260812 03:13:31.097821 25774 ts_tablet_manager.cc:1397] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 03:13:31.099287 25771 raft_consensus.cc:348] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d66658161d041af9b3dcfd548cb2e14" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37285 } } peers { permanent_uuid: "bd56af40dad645adb2c51e124e676d17" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35647 } } peers { permanent_uuid: "23d2dd3324d74e58b0c561a22ee68407" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42197 } }
I20260812 03:13:31.099520 25771 raft_consensus.cc:374] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:31.099591 25771 raft_consensus.cc:729] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d66658161d041af9b3dcfd548cb2e14, State: Initialized, Role: FOLLOWER
I20260812 03:13:31.099721 25778 tablet_bootstrap.cc:654] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:31.100085 25771 consensus_queue.cc:260] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d66658161d041af9b3dcfd548cb2e14" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37285 } } peers { permanent_uuid: "bd56af40dad645adb2c51e124e676d17" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35647 } } peers { permanent_uuid: "23d2dd3324d74e58b0c561a22ee68407" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42197 } }
I20260812 03:13:31.100770 25740 heartbeater.cc:503] Master 127.16.217.126:38227 was elected leader, sending a full tablet report...
I20260812 03:13:31.101109 25778 log.cc:824] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:31.102134 25771 ts_tablet_manager.cc:1428] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260812 03:13:31.102556 25774 raft_consensus.cc:348] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d66658161d041af9b3dcfd548cb2e14" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37285 } } peers { permanent_uuid: "bd56af40dad645adb2c51e124e676d17" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35647 } } peers { permanent_uuid: "23d2dd3324d74e58b0c561a22ee68407" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42197 } }
I20260812 03:13:31.102789 25774 raft_consensus.cc:374] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:31.102856 25774 raft_consensus.cc:729] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23d2dd3324d74e58b0c561a22ee68407, State: Initialized, Role: FOLLOWER
I20260812 03:13:31.103291 25774 consensus_queue.cc:260] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d66658161d041af9b3dcfd548cb2e14" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37285 } } peers { permanent_uuid: "bd56af40dad645adb2c51e124e676d17" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35647 } } peers { permanent_uuid: "23d2dd3324d74e58b0c561a22ee68407" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42197 } }
I20260812 03:13:31.103399 25778 tablet_bootstrap.cc:492] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17: No bootstrap required, opened a new log
I20260812 03:13:31.103955 25262 heartbeater.cc:503] Master 127.16.217.126:38227 was elected leader, sending a full tablet report...
I20260812 03:13:31.104244 25774 ts_tablet_manager.cc:1428] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 03:13:31.104872 25778 ts_tablet_manager.cc:1397] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 03:13:31.108498 25778 raft_consensus.cc:348] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d66658161d041af9b3dcfd548cb2e14" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37285 } } peers { permanent_uuid: "bd56af40dad645adb2c51e124e676d17" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35647 } } peers { permanent_uuid: "23d2dd3324d74e58b0c561a22ee68407" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42197 } }
I20260812 03:13:31.108716 25778 raft_consensus.cc:374] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:31.108759 25778 raft_consensus.cc:729] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd56af40dad645adb2c51e124e676d17, State: Initialized, Role: FOLLOWER
I20260812 03:13:31.109144 25778 consensus_queue.cc:260] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d66658161d041af9b3dcfd548cb2e14" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37285 } } peers { permanent_uuid: "bd56af40dad645adb2c51e124e676d17" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35647 } } peers { permanent_uuid: "23d2dd3324d74e58b0c561a22ee68407" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42197 } }
I20260812 03:13:31.109752 25485 heartbeater.cc:503] Master 127.16.217.126:38227 was elected leader, sending a full tablet report...
I20260812 03:13:31.110014 25778 ts_tablet_manager.cc:1428] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260812 03:13:31.268286 25742 tablet.cc:2369] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:13:31.283303 25486 tablet.cc:2369] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:13:31.309762 25268 tablet.cc:2369] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:13:31.402511 25786 raft_consensus.cc:482] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:31.402835 25786 raft_consensus.cc:504] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d66658161d041af9b3dcfd548cb2e14" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37285 } } peers { permanent_uuid: "bd56af40dad645adb2c51e124e676d17" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35647 } } peers { permanent_uuid: "23d2dd3324d74e58b0c561a22ee68407" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42197 } }
I20260812 03:13:31.404042 25786 leader_election.cc:288] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d66658161d041af9b3dcfd548cb2e14 (127.16.217.67:37285), bd56af40dad645adb2c51e124e676d17 (127.16.217.66:35647)
I20260812 03:13:31.409744 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:31.404563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:44917 (local address 127.16.217.67:37285)
0812 03:13:31.404801 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:31.404806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:31.404967 (+ 161us) server_negotiation.cc:407] Connection header received
0812 03:13:31.405157 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:31.405160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:31.405213 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:31.405297 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:31.405689 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.406536 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.408373 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.409012 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.409135 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:31.409192 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:31.409525 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 03:13:31.409602 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260812 03:13:31.409744 25767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:31.405082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:36133 (local address 127.16.217.66:35647)
0812 03:13:31.405256 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:31.405261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:31.405432 (+ 171us) server_negotiation.cc:407] Connection header received
0812 03:13:31.405531 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:31.405533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:31.405570 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:31.405670 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:31.405909 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.406656 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.408361 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.409003 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.409141 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:31.409192 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:31.409525 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 03:13:31.409602 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 03:13:31.409797 25794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:31.404940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:35647 (local address 127.16.217.65:36133)
0812 03:13:31.405321 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:31.405334 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:31.405450 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:31.405683 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:31.405685 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:31.405711 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:31.405803 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:31.405807 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.406770 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.406775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:31.408213 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:31.408225 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.409143 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.409147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:31.409313 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:31.409398 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 03:13:31.409483 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 03:13:31.409602 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":158,"spinlock_wait_cycles":384,"thread_start_us":97,"threads_started":1}
I20260812 03:13:31.409797 25788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:31.404403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:37285 (local address 127.16.217.65:44917)
0812 03:13:31.404856 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:31.404870 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:31.405024 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:31.405343 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:31.405347 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:31.405377 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:31.405568 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:31.405575 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.406671 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.406674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:31.408215 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:31.408225 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.409143 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.409147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:31.409309 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:31.409398 (+ 89us) client_negotiation.cc:769] Sending connection context
0812 03:13:31.409483 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 03:13:31.409601 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":181,"spinlock_wait_cycles":31360,"thread_start_us":169,"threads_started":1}
I20260812 03:13:31.410513 25650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2407511eb2b416a807c56217aa1df59" candidate_uuid: "23d2dd3324d74e58b0c561a22ee68407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d66658161d041af9b3dcfd548cb2e14" is_pre_election: true
I20260812 03:13:31.410687 25417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2407511eb2b416a807c56217aa1df59" candidate_uuid: "23d2dd3324d74e58b0c561a22ee68407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd56af40dad645adb2c51e124e676d17" is_pre_election: true
I20260812 03:13:31.410866 25650 raft_consensus.cc:2454] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23d2dd3324d74e58b0c561a22ee68407 in term 0.
I20260812 03:13:31.411008 25417 raft_consensus.cc:2454] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23d2dd3324d74e58b0c561a22ee68407 in term 0.
I20260812 03:13:31.411368 25062 leader_election.cc:302] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23d2dd3324d74e58b0c561a22ee68407, 9d66658161d041af9b3dcfd548cb2e14; no voters:
I20260812 03:13:31.411646 25786 raft_consensus.cc:2781] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:13:31.411751 25786 raft_consensus.cc:482] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:13:31.411834 25786 raft_consensus.cc:3037] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:31.412909 25786 raft_consensus.cc:504] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d66658161d041af9b3dcfd548cb2e14" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37285 } } peers { permanent_uuid: "bd56af40dad645adb2c51e124e676d17" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35647 } } peers { permanent_uuid: "23d2dd3324d74e58b0c561a22ee68407" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42197 } }
I20260812 03:13:31.413267 25786 leader_election.cc:288] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [CANDIDATE]: Term 1 election: Requested vote from peers 9d66658161d041af9b3dcfd548cb2e14 (127.16.217.67:37285), bd56af40dad645adb2c51e124e676d17 (127.16.217.66:35647)
I20260812 03:13:31.413774 25650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2407511eb2b416a807c56217aa1df59" candidate_uuid: "23d2dd3324d74e58b0c561a22ee68407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d66658161d041af9b3dcfd548cb2e14"
I20260812 03:13:31.413779 25417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2407511eb2b416a807c56217aa1df59" candidate_uuid: "23d2dd3324d74e58b0c561a22ee68407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd56af40dad645adb2c51e124e676d17"
I20260812 03:13:31.413908 25417 raft_consensus.cc:3037] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:31.413908 25650 raft_consensus.cc:3037] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:31.415262 25417 raft_consensus.cc:2454] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23d2dd3324d74e58b0c561a22ee68407 in term 1.
I20260812 03:13:31.415262 25650 raft_consensus.cc:2454] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23d2dd3324d74e58b0c561a22ee68407 in term 1.
I20260812 03:13:31.415640 25062 leader_election.cc:302] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23d2dd3324d74e58b0c561a22ee68407, bd56af40dad645adb2c51e124e676d17; no voters:
I20260812 03:13:31.415804 25786 raft_consensus.cc:2781] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:31.416102 25786 raft_consensus.cc:686] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [term 1 LEADER]: Becoming Leader. State: Replica: 23d2dd3324d74e58b0c561a22ee68407, State: Running, Role: LEADER
I20260812 03:13:31.416457 25786 consensus_queue.cc:237] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d66658161d041af9b3dcfd548cb2e14" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37285 } } peers { permanent_uuid: "bd56af40dad645adb2c51e124e676d17" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35647 } } peers { permanent_uuid: "23d2dd3324d74e58b0c561a22ee68407" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42197 } }
I20260812 03:13:31.419955 24913 catalog_manager.cc:5495] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23d2dd3324d74e58b0c561a22ee68407 (127.16.217.65). New cstate: current_term: 1 leader_uuid: "23d2dd3324d74e58b0c561a22ee68407" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d66658161d041af9b3dcfd548cb2e14" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd56af40dad645adb2c51e124e676d17" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23d2dd3324d74e58b0c561a22ee68407" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42197 } health_report { overall_health: HEALTHY } } }
I20260812 03:13:31.446859 25770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:31.443137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39906 (local address 127.16.217.65:42197)
0812 03:13:31.443282 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:31.443287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:31.443308 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:13:31.443399 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:31.443402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:31.443469 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:31.443586 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:31.443860 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.444379 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.445286 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:31.445521 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:31.445643 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:31.445969 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:31.446430 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:31.446571 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:31.446645 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 03:13:31.446704 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 03:13:31.453269 25650 raft_consensus.cc:1261] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14 [term 1 FOLLOWER]: Refusing update from remote peer 23d2dd3324d74e58b0c561a22ee68407: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:31.453269 25417 raft_consensus.cc:1261] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17 [term 1 FOLLOWER]: Refusing update from remote peer 23d2dd3324d74e58b0c561a22ee68407: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:31.454041 25786 consensus_queue.cc:1035] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd56af40dad645adb2c51e124e676d17" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:31.454321 25795 consensus_queue.cc:1035] T b2407511eb2b416a807c56217aa1df59 P 23d2dd3324d74e58b0c561a22ee68407 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d66658161d041af9b3dcfd548cb2e14" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:31.463375 25801 mvcc.cc:205] Tried to move back new op lower bound from 7317522069303750656 to 7317522069164916736. Current Snapshot: MvccSnapshot[applied={T|T < 7317522069303750656}]
I20260812 03:13:31.471413 24911 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:31.471606 24911 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:31.474085 24911 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 23da92fedf944408b1753d2ab719ab1f: Sending DeleteTablet for 3 replicas of tablet b2407511eb2b416a807c56217aa1df59
I20260812 03:13:31.475026 25617 tablet_service.cc:1514] Processing DeleteTablet for tablet b2407511eb2b416a807c56217aa1df59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:31 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:54464
I20260812 03:13:31.475378 25378 tablet_service.cc:1514] Processing DeleteTablet for tablet b2407511eb2b416a807c56217aa1df59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:31 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:53984
I20260812 03:13:31.475528 25816 tablet_replica.cc:330] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14: stopping tablet replica
I20260812 03:13:31.475715 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 25036
I20260812 03:13:31.485256 25816 raft_consensus.cc:2229] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:31.485710 25816 raft_consensus.cc:2258] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:31.486084 25818 tablet_replica.cc:330] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17: stopping tablet replica
I20260812 03:13:31.486579 25818 raft_consensus.cc:2229] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:31.486907 25818 raft_consensus.cc:2258] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:31.487891 25816 ts_tablet_manager.cc:1905] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:31.489423 25818 ts_tablet_manager.cc:1905] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:31.492427 25818 ts_tablet_manager.cc:1918] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:13:31.492604 25818 log.cc:1196] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/b2407511eb2b416a807c56217aa1df59
I20260812 03:13:31.492892 25816 ts_tablet_manager.cc:1918] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:13:31.492986 25818 ts_tablet_manager.cc:1939] T b2407511eb2b416a807c56217aa1df59 P bd56af40dad645adb2c51e124e676d17: Deleting consensus metadata
I20260812 03:13:31.493008 25816 log.cc:1196] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/b2407511eb2b416a807c56217aa1df59
I20260812 03:13:31.493320 25816 ts_tablet_manager.cc:1939] T b2407511eb2b416a807c56217aa1df59 P 9d66658161d041af9b3dcfd548cb2e14: Deleting consensus metadata
I20260812 03:13:31.494503 24884 catalog_manager.cc:4841] TS bd56af40dad645adb2c51e124e676d17 (127.16.217.66:35647): tablet b2407511eb2b416a807c56217aa1df59 (table test-table [id=4013212299b341268f476d1184e4e740]) successfully deleted
I20260812 03:13:31.494994 24884 catalog_manager.cc:4841] TS 9d66658161d041af9b3dcfd548cb2e14 (127.16.217.67:37285): tablet b2407511eb2b416a807c56217aa1df59 (table test-table [id=4013212299b341268f476d1184e4e740]) successfully deleted
W20260812 03:13:31.499473 24884 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.217.65:42197 (error 108)
I20260812 03:13:31.499744 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 25273
I20260812 03:13:31.501135 25769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:31.500716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:42197 (local address 127.0.0.1:39912)
0812 03:13:31.500964 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:31.501033 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:42197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
W20260812 03:13:31.501295 24884 catalog_manager.cc:4584] TS 23d2dd3324d74e58b0c561a22ee68407 (127.16.217.65:42197): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b2407511eb2b416a807c56217aa1df59: Network error: Client connection negotiation failed: client connection to 127.16.217.65:42197: connect: Connection refused (error 111)
I20260812 03:13:31.519472 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 25492
I20260812 03:13:31.533891 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 24861
2026-08-12T03:13:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2903 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T03:13:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:31Z Disabled control of system clock
I20260812 03:13:31.577474 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:39727
--webserver_interface=127.16.217.126
--webserver_port=0
--builtin_ntp_servers=127.16.217.84:44995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:39727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 03:13:31.691794 25826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:31.692095 25826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:31.692133 25826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:31.696281 25826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:31.696349 25826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:31.696365 25826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:31.696381 25826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:31.696396 25826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:13:31.701136 25826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:44995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:39727
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:39727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:31.702301 25826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:31.703825 25826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:31.712695 25847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:31.712838 25839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:31.712783 25840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:31.713042 25826 server_base.cc:1034] running on GCE node
I20260812 03:13:31.714365 25826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:31.716547 25826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:31.717739 25826 hybrid_clock.cc:648] HybridClock initialized: now 1786504411717706 us; error 50 us; skew 500 ppm
I20260812 03:13:31.720021 25826 webserver.cc:460] Webserver started at http://127.16.217.126:39461/ using document root <none> and password file <none>
I20260812 03:13:31.720651 25826 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:31.720726 25826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:31.720968 25826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:31.723302 25826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "44e0c139c7c0416ab6b099a7354b2cbc"
format_stamp: "Formatted at 2026-08-12 03:13:31 on dist-test-slave-7nm7"
server_key: "03d1a8bd510401ed50644efb790103c1"
server_key_iv: "ec912a11928d74f40c3dfe97b8261f66"
server_key_version: "encryptionkey@0"
I20260812 03:13:31.723893 25826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "44e0c139c7c0416ab6b099a7354b2cbc"
format_stamp: "Formatted at 2026-08-12 03:13:31 on dist-test-slave-7nm7"
server_key: "03d1a8bd510401ed50644efb790103c1"
server_key_iv: "ec912a11928d74f40c3dfe97b8261f66"
server_key_version: "encryptionkey@0"
I20260812 03:13:31.727857 25826 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:13:31.730545 25856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:31.731959 25826 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:31.732161 25826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "44e0c139c7c0416ab6b099a7354b2cbc"
format_stamp: "Formatted at 2026-08-12 03:13:31 on dist-test-slave-7nm7"
server_key: "03d1a8bd510401ed50644efb790103c1"
server_key_iv: "ec912a11928d74f40c3dfe97b8261f66"
server_key_version: "encryptionkey@0"
I20260812 03:13:31.732306 25826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:31.737594 25826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:31.738348 25826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:31.738543 25826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:31.746308 25826 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:39727
I20260812 03:13:31.746374 25949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:39727 every 8 connection(s)
I20260812 03:13:31.747316 25826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:31.750813 25950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:31.753830 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 25826
I20260812 03:13:31.753955 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:13:31.754262 17253 external_mini_cluster.cc:1428] Setting key 29fb82977b2e2bc77a4e64d1532b29eb
I20260812 03:13:31.757386 25950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc: Bootstrap starting.
I20260812 03:13:31.759955 25950 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:31.760241 25956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:31.755884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49090 (local address 127.16.217.126:39727)
0812 03:13:31.756629 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:31.756644 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:31.756685 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:13:31.757309 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:31.757326 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:31.757726 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:31.758057 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:31.758880 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:31.758885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:31.758897 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:13:31.758906 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:31.758958 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:13:31.758973 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:31.759087 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:31.759212 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 03:13:31.759484 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":163,"threads_started":1}
I20260812 03:13:31.760949 25950 log.cc:824] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:31.763156 25950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc: No bootstrap required, opened a new log
I20260812 03:13:31.767051 25950 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44e0c139c7c0416ab6b099a7354b2cbc" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39727 } }
I20260812 03:13:31.767287 25950 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:31.767333 25950 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44e0c139c7c0416ab6b099a7354b2cbc, State: Initialized, Role: FOLLOWER
I20260812 03:13:31.767746 25950 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44e0c139c7c0416ab6b099a7354b2cbc" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39727 } }
I20260812 03:13:31.767874 25950 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:31.767966 25950 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:31.768100 25950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:31.769097 25950 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44e0c139c7c0416ab6b099a7354b2cbc" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39727 } }
I20260812 03:13:31.769405 25950 leader_election.cc:302] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44e0c139c7c0416ab6b099a7354b2cbc; no voters:
I20260812 03:13:31.769665 25950 leader_election.cc:288] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:13:31.769824 25958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:31.770097 25958 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc [term 1 LEADER]: Becoming Leader. State: Replica: 44e0c139c7c0416ab6b099a7354b2cbc, State: Running, Role: LEADER
I20260812 03:13:31.770447 25958 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44e0c139c7c0416ab6b099a7354b2cbc" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39727 } }
I20260812 03:13:31.770958 25950 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:31.772236 25960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44e0c139c7c0416ab6b099a7354b2cbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44e0c139c7c0416ab6b099a7354b2cbc" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39727 } } }
I20260812 03:13:31.772274 25959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44e0c139c7c0416ab6b099a7354b2cbc. Latest consensus state: current_term: 1 leader_uuid: "44e0c139c7c0416ab6b099a7354b2cbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44e0c139c7c0416ab6b099a7354b2cbc" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39727 } } }
I20260812 03:13:31.772375 25959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:31.772377 25960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:31.772795 25983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:31.776010 25983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:31.781888 25983 catalog_manager.cc:1347] Generated new cluster ID: d627b83d046345aaadc57d79785b7365
I20260812 03:13:31.781984 25983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:31.793838 25983 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:13:31.794399 25983 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:31.802748 25983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc: Generated new TSK 0
I20260812 03:13:31.803609 25983 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:13:31.814414 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39727
--builtin_ntp_servers=127.16.217.84:44995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 03:13:31.943926 26000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:31.944195 26000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:31.944235 26000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:31.948334 26000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:31.948385 26000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:31.948473 26000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:13:31.953339 26000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:44995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:31.954496 26000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:31.955897 26000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:31.965493 26006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:31.965567 26007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:31.965507 26015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:31.965694 26000 server_base.cc:1034] running on GCE node
I20260812 03:13:31.966274 26000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:31.966918 26000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:31.968170 26000 hybrid_clock.cc:648] HybridClock initialized: now 1786504411968147 us; error 38 us; skew 500 ppm
I20260812 03:13:31.970031 26000 webserver.cc:460] Webserver started at http://127.16.217.65:35897/ using document root <none> and password file <none>
I20260812 03:13:31.970695 26000 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:31.970793 26000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:31.971036 26000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:31.973291 26000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "56685a979f90401bba308f0e40ec13e3"
format_stamp: "Formatted at 2026-08-12 03:13:31 on dist-test-slave-7nm7"
server_key: "7f025460018fdf4cefd9776760e4947d"
server_key_iv: "b338c191f0060b31326efa502608403b"
server_key_version: "encryptionkey@0"
I20260812 03:13:31.973846 26000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "56685a979f90401bba308f0e40ec13e3"
format_stamp: "Formatted at 2026-08-12 03:13:31 on dist-test-slave-7nm7"
server_key: "7f025460018fdf4cefd9776760e4947d"
server_key_iv: "b338c191f0060b31326efa502608403b"
server_key_version: "encryptionkey@0"
I20260812 03:13:31.977587 26000 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 03:13:31.980346 26023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:31.981662 26000 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:13:31.981818 26000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "56685a979f90401bba308f0e40ec13e3"
format_stamp: "Formatted at 2026-08-12 03:13:31 on dist-test-slave-7nm7"
server_key: "7f025460018fdf4cefd9776760e4947d"
server_key_iv: "b338c191f0060b31326efa502608403b"
server_key_version: "encryptionkey@0"
I20260812 03:13:31.981945 26000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:31.986647 26000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:31.987277 26000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:31.987461 26000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:31.988011 26000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:31.988952 26000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:31.989023 26000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:31.989087 26000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:31.989135 26000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:31.997802 26000 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:33973
I20260812 03:13:31.997824 26219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:33973 every 8 connection(s)
I20260812 03:13:31.998749 26000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:13:32.002946 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 26000
I20260812 03:13:32.003114 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:13:32.003424 17253 external_mini_cluster.cc:1428] Setting key 55287e4a2ba5f566c5f35d4d4acebe57
I20260812 03:13:32.005498 25956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.000622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:53309 (local address 127.16.217.126:39727)
0812 03:13:32.000804 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:32.000809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:32.001845 (+ 1036us) server_negotiation.cc:407] Connection header received
0812 03:13:32.003562 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:32.003566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:32.003626 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:32.003706 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:32.004980 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:32.004983 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:32.004985 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:13:32.004997 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:32.005017 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:13:32.005025 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:32.005152 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:32.005320 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 03:13:32.005367 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 03:13:32.005926 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39727
--builtin_ntp_servers=127.16.217.84:44995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 03:13:32.006542 26225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.001033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39727 (local address 127.16.217.65:53309)
0812 03:13:32.001702 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:32.001741 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:32.003293 (+ 1552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:32.003914 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:32.003920 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:32.004006 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:32.004658 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:13:32.004660 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:32.004735 (+ 75us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:13:32.004819 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:32.005161 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:32.005170 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:32.005175 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:13:32.005365 (+ 190us) client_negotiation.cc:240] Negotiation successful
0812 03:13:32.005611 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":125,"threads_started":1}
I20260812 03:13:32.007906 26220 heartbeater.cc:347] Connected to a master server at 127.16.217.126:39727
I20260812 03:13:32.008188 26220 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:32.008703 26220 heartbeater.cc:511] Master 127.16.217.126:39727 requested a full tablet report, sending...
I20260812 03:13:32.010499 25890 ts_manager.cc:194] Registered new tserver with Master: 56685a979f90401bba308f0e40ec13e3 (127.16.217.65:33973)
W20260812 03:13:32.127702 26226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:32.127987 26226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:32.128028 26226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:32.132238 26226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:32.132310 26226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:32.132371 26226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:13:32.137526 26226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:44995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:32.138827 26226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:32.140321 26226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:32.150390 26234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:32.150400 26237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:32.150504 26226 server_base.cc:1034] running on GCE node
W20260812 03:13:32.150390 26235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:32.151075 26226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:32.151749 26226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:32.152961 26226 hybrid_clock.cc:648] HybridClock initialized: now 1786504412152947 us; error 35 us; skew 500 ppm
I20260812 03:13:32.155076 26226 webserver.cc:460] Webserver started at http://127.16.217.66:45569/ using document root <none> and password file <none>
I20260812 03:13:32.155699 26226 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:32.155764 26226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:32.156005 26226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:32.158154 26226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "875290aa5dea40a1a6370074aa72e6b2"
format_stamp: "Formatted at 2026-08-12 03:13:32 on dist-test-slave-7nm7"
server_key: "b4669b0e929af62e64b8185465669a3f"
server_key_iv: "218a7496b14e36924c338e7cd6e7ae05"
server_key_version: "encryptionkey@0"
I20260812 03:13:32.158828 26226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "875290aa5dea40a1a6370074aa72e6b2"
format_stamp: "Formatted at 2026-08-12 03:13:32 on dist-test-slave-7nm7"
server_key: "b4669b0e929af62e64b8185465669a3f"
server_key_iv: "218a7496b14e36924c338e7cd6e7ae05"
server_key_version: "encryptionkey@0"
I20260812 03:13:32.162861 26226 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:13:32.166150 26251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:32.167795 26226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:32.167959 26226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "875290aa5dea40a1a6370074aa72e6b2"
format_stamp: "Formatted at 2026-08-12 03:13:32 on dist-test-slave-7nm7"
server_key: "b4669b0e929af62e64b8185465669a3f"
server_key_iv: "218a7496b14e36924c338e7cd6e7ae05"
server_key_version: "encryptionkey@0"
I20260812 03:13:32.168099 26226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:32.173435 26226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:32.174149 26226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:32.174407 26226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:32.175014 26226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:32.176853 26226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:32.176949 26226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:32.177165 26226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:32.177320 26226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:32.187325 26226 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:41307
I20260812 03:13:32.187350 26454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:41307 every 8 connection(s)
I20260812 03:13:32.188301 26226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:13:32.194589 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 26226
I20260812 03:13:32.194736 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:13:32.195024 17253 external_mini_cluster.cc:1428] Setting key 9e4cb124b8b0dc044e92327e4f4cb015
I20260812 03:13:32.195173 25956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.190642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:48417 (local address 127.16.217.126:39727)
0812 03:13:32.190795 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:32.190798 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:32.191849 (+ 1051us) server_negotiation.cc:407] Connection header received
0812 03:13:32.193256 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:32.193260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:32.193309 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:32.193423 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:32.194641 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:32.194645 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:32.194648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:13:32.194666 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:32.194691 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:13:32.194702 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:32.194795 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:32.194976 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 03:13:32.195026 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:13:32.196224 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.190987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39727 (local address 127.16.217.66:48417)
0812 03:13:32.191687 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:32.191729 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:32.193006 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:32.193605 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:32.193613 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:32.193699 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:32.194364 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:13:32.194367 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:32.194409 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:13:32.194482 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:32.194810 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:32.194822 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:32.194828 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:13:32.195044 (+ 216us) client_negotiation.cc:240] Negotiation successful
0812 03:13:32.195310 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":128,"threads_started":1}
I20260812 03:13:32.197407 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39727
--builtin_ntp_servers=127.16.217.84:44995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 03:13:32.197522 26462 heartbeater.cc:347] Connected to a master server at 127.16.217.126:39727
I20260812 03:13:32.197751 26462 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:32.198338 26462 heartbeater.cc:511] Master 127.16.217.126:39727 requested a full tablet report, sending...
I20260812 03:13:32.199509 25890 ts_manager.cc:194] Registered new tserver with Master: 875290aa5dea40a1a6370074aa72e6b2 (127.16.217.66:41307)
W20260812 03:13:32.321302 26466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:32.321564 26466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:32.321604 26466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:32.325579 26466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:32.325631 26466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:32.325686 26466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:13:32.330310 26466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:44995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:32.331362 26466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:32.332721 26466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:32.342731 26479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:32.342733 26473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:32.342897 26466 server_base.cc:1034] running on GCE node
W20260812 03:13:32.342733 26477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:32.343528 26466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:32.344218 26466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:32.345412 26466 hybrid_clock.cc:648] HybridClock initialized: now 1786504412345390 us; error 35 us; skew 500 ppm
I20260812 03:13:32.347406 26466 webserver.cc:460] Webserver started at http://127.16.217.67:35871/ using document root <none> and password file <none>
I20260812 03:13:32.348028 26466 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:32.348125 26466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:32.348349 26466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:32.350497 26466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "539a904adf304706ba4bb28d27da130d"
format_stamp: "Formatted at 2026-08-12 03:13:32 on dist-test-slave-7nm7"
server_key: "16735b66a20869a6135f25968caff734"
server_key_iv: "4e09883f4248abc046b2f8ad8de858d4"
server_key_version: "encryptionkey@0"
I20260812 03:13:32.351086 26466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "539a904adf304706ba4bb28d27da130d"
format_stamp: "Formatted at 2026-08-12 03:13:32 on dist-test-slave-7nm7"
server_key: "16735b66a20869a6135f25968caff734"
server_key_iv: "4e09883f4248abc046b2f8ad8de858d4"
server_key_version: "encryptionkey@0"
I20260812 03:13:32.354914 26466 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:13:32.357555 26487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:32.358896 26466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260812 03:13:32.359045 26466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "539a904adf304706ba4bb28d27da130d"
format_stamp: "Formatted at 2026-08-12 03:13:32 on dist-test-slave-7nm7"
server_key: "16735b66a20869a6135f25968caff734"
server_key_iv: "4e09883f4248abc046b2f8ad8de858d4"
server_key_version: "encryptionkey@0"
I20260812 03:13:32.359169 26466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:32.363698 26466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:32.364300 26466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:32.364495 26466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:32.365051 26466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:32.366005 26466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:32.366079 26466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:32.366169 26466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:32.366221 26466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:32.375774 26466 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:46457
I20260812 03:13:32.375795 26686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:46457 every 8 connection(s)
I20260812 03:13:32.376694 26466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:13:32.382908 25956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.378608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:49509 (local address 127.16.217.126:39727)
0812 03:13:32.378772 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:32.378777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:32.379919 (+ 1142us) server_negotiation.cc:407] Connection header received
0812 03:13:32.381211 (+ 1292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:32.381214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:32.381253 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:32.381331 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:32.382512 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:32.382515 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:32.382517 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:13:32.382529 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:32.382547 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:13:32.382555 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:32.382634 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:32.382761 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 03:13:32.382800 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:13:32.383786 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.379058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39727 (local address 127.16.217.67:49509)
0812 03:13:32.379764 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:32.379803 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:32.380997 (+ 1194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:32.381531 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:32.381538 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:32.381624 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:32.382180 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:13:32.382181 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:32.382220 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:13:32.382363 (+ 143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:32.382636 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:32.382643 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:32.382647 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:13:32.382777 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 03:13:32.382959 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":125,"threads_started":1}
I20260812 03:13:32.384963 26689 heartbeater.cc:347] Connected to a master server at 127.16.217.126:39727
I20260812 03:13:32.385223 26689 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:32.385708 26689 heartbeater.cc:511] Master 127.16.217.126:39727 requested a full tablet report, sending...
I20260812 03:13:32.385923 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 26466
I20260812 03:13:32.386060 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:13:32.386416 17253 external_mini_cluster.cc:1428] Setting key 3c59714c8822438c39750fbca685dd1e
I20260812 03:13:32.386983 25890 ts_manager.cc:194] Registered new tserver with Master: 539a904adf304706ba4bb28d27da130d (127.16.217.67:46457)
I20260812 03:13:32.388566 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:13:32.392294 25956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.390617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49102 (local address 127.16.217.126:39727)
0812 03:13:32.390768 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:32.390771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:32.390826 (+ 55us) server_negotiation.cc:407] Connection header received
0812 03:13:32.390951 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:32.390953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:32.390980 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:32.391063 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:32.391834 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:32.391837 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:32.391839 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:13:32.391849 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:32.391866 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:13:32.391873 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:32.391943 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:32.392068 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 03:13:32.392100 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 03:13:32.395112 25890 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:13:32.398077 25890 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:13:32.410591 26706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.406893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:46457 (local address 127.0.0.1:37962)
0812 03:13:32.408143 (+ 1250us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:32.408213 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:32.408317 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:32.409041 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:32.409047 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:32.409127 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:32.409740 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:13:32.409742 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:32.409766 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:13:32.409831 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:32.410177 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:32.410191 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:32.410200 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 03:13:32.410352 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 03:13:32.410426 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1069,"thread_start_us":91,"threads_started":1}
I20260812 03:13:32.410882 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.407731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37962 (local address 127.16.217.67:46457)
0812 03:13:32.408563 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:32.408577 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:32.408606 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:13:32.408665 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:32.408671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:32.408878 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:32.409036 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:32.409978 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:32.409984 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:32.409997 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:13:32.410008 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:32.410051 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:13:32.410080 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:32.410203 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:32.410373 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 03:13:32.410448 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":119,"threads_started":1}
I20260812 03:13:32.411834 26707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.407052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:41307 (local address 127.0.0.1:56106)
0812 03:13:32.408523 (+ 1471us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:32.408536 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:32.408733 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:32.409811 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:32.409814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:32.409828 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:32.411099 (+ 1271us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:13:32.411103 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:32.411117 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:13:32.411145 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:32.411472 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:32.411476 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:32.411479 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:13:32.411667 (+ 188us) client_negotiation.cc:240] Negotiation successful
0812 03:13:32.411706 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1405,"thread_start_us":93,"threads_started":1}
I20260812 03:13:32.411986 26711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.407586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56106 (local address 127.16.217.66:41307)
0812 03:13:32.407987 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:32.408003 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:32.408670 (+ 667us) server_negotiation.cc:407] Connection header received
0812 03:13:32.408867 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:32.408874 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:32.409053 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:32.409192 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:32.411247 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:32.411252 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:32.411286 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:13:32.411300 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:32.411342 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:13:32.411373 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:32.411486 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:32.411752 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 03:13:32.411842 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":131,"threads_started":1}
I20260812 03:13:32.412671 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.408514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:33973 (local address 127.0.0.1:36138)
0812 03:13:32.408742 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:32.408756 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:32.408820 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:32.410308 (+ 1488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:32.410311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:32.410325 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:32.410741 (+ 416us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:13:32.410744 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:32.410755 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:13:32.410782 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:32.412398 (+ 1616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:32.412401 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:32.412405 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:13:32.412503 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 03:13:32.412538 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":98,"threads_started":1}
I20260812 03:13:32.412765 26714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.408920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36138 (local address 127.16.217.65:33973)
0812 03:13:32.409854 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:32.409862 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:32.409881 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:13:32.409959 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:32.409964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:32.410130 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:32.410290 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:32.412151 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:32.412157 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:32.412169 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:13:32.412180 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:32.412221 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:13:32.412252 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:32.412370 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:32.412534 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 03:13:32.412602 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":840,"thread_start_us":119,"threads_started":1}
I20260812 03:13:32.414011 26567 tablet_service.cc:1467] Processing CreateTablet for tablet 8255be28fb544d898425f886cb03fbaf (DEFAULT_TABLE table=test-table [id=60505a2fa90141ea89f2cf97918fe3d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:32.415226 26567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8255be28fb544d898425f886cb03fbaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:32.415175 26109 tablet_service.cc:1467] Processing CreateTablet for tablet 8255be28fb544d898425f886cb03fbaf (DEFAULT_TABLE table=test-table [id=60505a2fa90141ea89f2cf97918fe3d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:32.416172 26109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8255be28fb544d898425f886cb03fbaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:32.416170 26334 tablet_service.cc:1467] Processing CreateTablet for tablet 8255be28fb544d898425f886cb03fbaf (DEFAULT_TABLE table=test-table [id=60505a2fa90141ea89f2cf97918fe3d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:32.417125 26334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8255be28fb544d898425f886cb03fbaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:32.421838 26715 tablet_bootstrap.cc:492] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d: Bootstrap starting.
I20260812 03:13:32.423731 26717 tablet_bootstrap.cc:492] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3: Bootstrap starting.
I20260812 03:13:32.424263 26718 tablet_bootstrap.cc:492] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2: Bootstrap starting.
I20260812 03:13:32.425114 26715 tablet_bootstrap.cc:654] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:32.425783 26717 tablet_bootstrap.cc:654] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:32.426442 26715 log.cc:824] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:32.426601 26717 log.cc:824] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:32.426900 26718 tablet_bootstrap.cc:654] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:32.427717 26718 log.cc:824] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:32.428767 26715 tablet_bootstrap.cc:492] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d: No bootstrap required, opened a new log
I20260812 03:13:32.428901 26717 tablet_bootstrap.cc:492] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3: No bootstrap required, opened a new log
I20260812 03:13:32.428977 26715 ts_tablet_manager.cc:1397] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260812 03:13:32.429092 26717 ts_tablet_manager.cc:1397] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:13:32.429647 26718 tablet_bootstrap.cc:492] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2: No bootstrap required, opened a new log
I20260812 03:13:32.429847 26718 ts_tablet_manager.cc:1397] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:13:32.432794 26717 raft_consensus.cc:348] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "539a904adf304706ba4bb28d27da130d" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46457 } } peers { permanent_uuid: "875290aa5dea40a1a6370074aa72e6b2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41307 } } peers { permanent_uuid: "56685a979f90401bba308f0e40ec13e3" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33973 } }
I20260812 03:13:32.433085 26717 raft_consensus.cc:374] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:32.433154 26717 raft_consensus.cc:729] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56685a979f90401bba308f0e40ec13e3, State: Initialized, Role: FOLLOWER
I20260812 03:13:32.433621 26717 consensus_queue.cc:260] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "539a904adf304706ba4bb28d27da130d" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46457 } } peers { permanent_uuid: "875290aa5dea40a1a6370074aa72e6b2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41307 } } peers { permanent_uuid: "56685a979f90401bba308f0e40ec13e3" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33973 } }
I20260812 03:13:32.433828 26715 raft_consensus.cc:348] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "539a904adf304706ba4bb28d27da130d" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46457 } } peers { permanent_uuid: "875290aa5dea40a1a6370074aa72e6b2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41307 } } peers { permanent_uuid: "56685a979f90401bba308f0e40ec13e3" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33973 } }
I20260812 03:13:32.434096 26715 raft_consensus.cc:374] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:32.434172 26715 raft_consensus.cc:729] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 539a904adf304706ba4bb28d27da130d, State: Initialized, Role: FOLLOWER
I20260812 03:13:32.434662 26220 heartbeater.cc:503] Master 127.16.217.126:39727 was elected leader, sending a full tablet report...
I20260812 03:13:32.434713 26717 ts_tablet_manager.cc:1428] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 03:13:32.434741 26715 consensus_queue.cc:260] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "539a904adf304706ba4bb28d27da130d" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46457 } } peers { permanent_uuid: "875290aa5dea40a1a6370074aa72e6b2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41307 } } peers { permanent_uuid: "56685a979f90401bba308f0e40ec13e3" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33973 } }
I20260812 03:13:32.435384 26689 heartbeater.cc:503] Master 127.16.217.126:39727 was elected leader, sending a full tablet report...
I20260812 03:13:32.435664 26715 ts_tablet_manager.cc:1428] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260812 03:13:32.435791 26718 raft_consensus.cc:348] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "539a904adf304706ba4bb28d27da130d" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46457 } } peers { permanent_uuid: "875290aa5dea40a1a6370074aa72e6b2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41307 } } peers { permanent_uuid: "56685a979f90401bba308f0e40ec13e3" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33973 } }
I20260812 03:13:32.436034 26718 raft_consensus.cc:374] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:32.436095 26718 raft_consensus.cc:729] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 875290aa5dea40a1a6370074aa72e6b2, State: Initialized, Role: FOLLOWER
I20260812 03:13:32.436498 26718 consensus_queue.cc:260] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "539a904adf304706ba4bb28d27da130d" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46457 } } peers { permanent_uuid: "875290aa5dea40a1a6370074aa72e6b2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41307 } } peers { permanent_uuid: "56685a979f90401bba308f0e40ec13e3" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33973 } }
I20260812 03:13:32.437450 26718 ts_tablet_manager.cc:1428] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260812 03:13:32.437669 26462 heartbeater.cc:503] Master 127.16.217.126:39727 was elected leader, sending a full tablet report...
W20260812 03:13:32.439322 26464 tablet.cc:2369] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:13:32.499972 26223 tablet.cc:2369] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:13:32.552261 26733 raft_consensus.cc:482] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:32.552546 26733 raft_consensus.cc:504] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "539a904adf304706ba4bb28d27da130d" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46457 } } peers { permanent_uuid: "875290aa5dea40a1a6370074aa72e6b2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41307 } } peers { permanent_uuid: "56685a979f90401bba308f0e40ec13e3" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33973 } }
I20260812 03:13:32.553946 26733 leader_election.cc:288] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 539a904adf304706ba4bb28d27da130d (127.16.217.67:46457), 875290aa5dea40a1a6370074aa72e6b2 (127.16.217.66:41307)
I20260812 03:13:32.556342 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.554270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:41307 (local address 127.16.217.65:58619)
0812 03:13:32.554637 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:32.554649 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:32.554780 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:32.555091 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:32.555094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:32.555109 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:32.555641 (+ 532us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:13:32.555644 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:32.555658 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:13:32.555696 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:32.555996 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:32.556007 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:32.556010 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:13:32.556108 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 03:13:32.556153 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":110,"threads_started":1}
I20260812 03:13:32.556402 26711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.554370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:58619 (local address 127.16.217.66:41307)
0812 03:13:32.554506 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:32.554511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:32.554735 (+ 224us) server_negotiation.cc:407] Connection header received
0812 03:13:32.554891 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:32.554894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:32.554939 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:32.555073 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:32.555840 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:32.555844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:32.555847 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:13:32.555861 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:32.555882 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:13:32.555895 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:32.555993 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:32.556203 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 03:13:32.556257 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:13:32.556490 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.554450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:36087 (local address 127.16.217.67:46457)
0812 03:13:32.554592 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:32.554601 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:32.554842 (+ 241us) server_negotiation.cc:407] Connection header received
0812 03:13:32.554979 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:32.554983 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:32.555025 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:32.555123 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:32.556006 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:32.556010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:32.556012 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:13:32.556024 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:32.556047 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:13:32.556061 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:32.556147 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:32.556317 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 03:13:32.556363 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:13:32.556529 26739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.554335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:46457 (local address 127.16.217.65:36087)
0812 03:13:32.554746 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:32.554758 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:32.554870 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:32.555147 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:32.555150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:32.555165 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:32.555823 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:13:32.555825 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:32.555836 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:13:32.555862 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:32.556193 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:32.556196 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:32.556200 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:13:32.556305 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 03:13:32.556339 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":66,"threads_started":1}
I20260812 03:13:32.557144 26606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8255be28fb544d898425f886cb03fbaf" candidate_uuid: "56685a979f90401bba308f0e40ec13e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "539a904adf304706ba4bb28d27da130d" is_pre_election: true
I20260812 03:13:32.557142 26368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8255be28fb544d898425f886cb03fbaf" candidate_uuid: "56685a979f90401bba308f0e40ec13e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "875290aa5dea40a1a6370074aa72e6b2" is_pre_election: true
I20260812 03:13:32.557485 26606 raft_consensus.cc:2454] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56685a979f90401bba308f0e40ec13e3 in term 0.
I20260812 03:13:32.557485 26368 raft_consensus.cc:2454] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56685a979f90401bba308f0e40ec13e3 in term 0.
I20260812 03:13:32.558111 26031 leader_election.cc:302] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 539a904adf304706ba4bb28d27da130d, 56685a979f90401bba308f0e40ec13e3; no voters:
I20260812 03:13:32.558458 26733 raft_consensus.cc:2781] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:13:32.558583 26733 raft_consensus.cc:482] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:13:32.558645 26733 raft_consensus.cc:3037] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:32.559770 26733 raft_consensus.cc:504] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "539a904adf304706ba4bb28d27da130d" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46457 } } peers { permanent_uuid: "875290aa5dea40a1a6370074aa72e6b2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41307 } } peers { permanent_uuid: "56685a979f90401bba308f0e40ec13e3" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33973 } }
I20260812 03:13:32.560168 26733 leader_election.cc:288] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [CANDIDATE]: Term 1 election: Requested vote from peers 539a904adf304706ba4bb28d27da130d (127.16.217.67:46457), 875290aa5dea40a1a6370074aa72e6b2 (127.16.217.66:41307)
I20260812 03:13:32.560607 26368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8255be28fb544d898425f886cb03fbaf" candidate_uuid: "56685a979f90401bba308f0e40ec13e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "875290aa5dea40a1a6370074aa72e6b2"
I20260812 03:13:32.560611 26606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8255be28fb544d898425f886cb03fbaf" candidate_uuid: "56685a979f90401bba308f0e40ec13e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "539a904adf304706ba4bb28d27da130d"
I20260812 03:13:32.560750 26606 raft_consensus.cc:3037] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:32.560750 26368 raft_consensus.cc:3037] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:32.562407 26606 raft_consensus.cc:2454] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56685a979f90401bba308f0e40ec13e3 in term 1.
I20260812 03:13:32.562407 26368 raft_consensus.cc:2454] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56685a979f90401bba308f0e40ec13e3 in term 1.
I20260812 03:13:32.562702 26031 leader_election.cc:302] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 539a904adf304706ba4bb28d27da130d, 56685a979f90401bba308f0e40ec13e3; no voters:
I20260812 03:13:32.562924 26733 raft_consensus.cc:2781] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:32.563169 26733 raft_consensus.cc:686] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [term 1 LEADER]: Becoming Leader. State: Replica: 56685a979f90401bba308f0e40ec13e3, State: Running, Role: LEADER
I20260812 03:13:32.563553 26733 consensus_queue.cc:237] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "539a904adf304706ba4bb28d27da130d" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46457 } } peers { permanent_uuid: "875290aa5dea40a1a6370074aa72e6b2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41307 } } peers { permanent_uuid: "56685a979f90401bba308f0e40ec13e3" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33973 } }
I20260812 03:13:32.567374 25890 catalog_manager.cc:5495] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56685a979f90401bba308f0e40ec13e3 (127.16.217.65). New cstate: current_term: 1 leader_uuid: "56685a979f90401bba308f0e40ec13e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "539a904adf304706ba4bb28d27da130d" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "875290aa5dea40a1a6370074aa72e6b2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56685a979f90401bba308f0e40ec13e3" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33973 } health_report { overall_health: HEALTHY } } }
I20260812 03:13:32.604620 26714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.602901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36154 (local address 127.16.217.65:33973)
0812 03:13:32.603081 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:32.603087 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:32.603105 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:13:32.603196 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:32.603198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:32.603263 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:32.603363 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:32.604145 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:32.604149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:32.604151 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:13:32.604173 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:32.604194 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:13:32.604202 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:32.604325 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:32.604418 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 03:13:32.604460 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 03:13:32.611325 26368 raft_consensus.cc:1261] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2 [term 1 FOLLOWER]: Refusing update from remote peer 56685a979f90401bba308f0e40ec13e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:32.611443 26606 raft_consensus.cc:1261] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d [term 1 FOLLOWER]: Refusing update from remote peer 56685a979f90401bba308f0e40ec13e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:32.612179 26733 consensus_queue.cc:1035] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "875290aa5dea40a1a6370074aa72e6b2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:32.612396 26740 consensus_queue.cc:1035] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "539a904adf304706ba4bb28d27da130d" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:32.621428 26749 mvcc.cc:205] Tried to move back new op lower bound from 7317522074047393792 to 7317522073862955008. Current Snapshot: MvccSnapshot[applied={T|T < 7317522074047393792}]
I20260812 03:13:32.630748 25890 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:32.630926 25890 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:32.633772 25890 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44e0c139c7c0416ab6b099a7354b2cbc: Sending DeleteTablet for 3 replicas of tablet 8255be28fb544d898425f886cb03fbaf
I20260812 03:13:32.634791 26334 tablet_service.cc:1514] Processing DeleteTablet for tablet 8255be28fb544d898425f886cb03fbaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:32 UTC) from {username='slave'} at 127.0.0.1:56106
I20260812 03:13:32.634791 26567 tablet_service.cc:1514] Processing DeleteTablet for tablet 8255be28fb544d898425f886cb03fbaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:32 UTC) from {username='slave'} at 127.0.0.1:37962
I20260812 03:13:32.635041 26109 tablet_service.cc:1514] Processing DeleteTablet for tablet 8255be28fb544d898425f886cb03fbaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:32 UTC) from {username='slave'} at 127.0.0.1:36138
I20260812 03:13:32.635577 26766 tablet_replica.cc:330] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3: stopping tablet replica
I20260812 03:13:32.635773 26765 tablet_replica.cc:330] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d: stopping tablet replica
I20260812 03:13:32.635977 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 26000
I20260812 03:13:32.636113 26765 raft_consensus.cc:2229] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:32.636169 26766 raft_consensus.cc:2229] T 8255be28fb544d898425f886cb03fbaf P 56685a979f90401bba308f0e40ec13e3 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:13:32.636445 26765 raft_consensus.cc:2258] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:32.636963 26764 tablet_replica.cc:330] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2: stopping tablet replica
I20260812 03:13:32.637292 26764 raft_consensus.cc:2229] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:32.637552 26764 raft_consensus.cc:2258] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:32.641228 26764 ts_tablet_manager.cc:1905] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:32.641230 26765 ts_tablet_manager.cc:1905] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:32.644429 26764 ts_tablet_manager.cc:1918] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:13:32.644429 26765 ts_tablet_manager.cc:1918] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:13:32.644551 26764 log.cc:1196] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/8255be28fb544d898425f886cb03fbaf
I20260812 03:13:32.644551 26765 log.cc:1196] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/8255be28fb544d898425f886cb03fbaf
I20260812 03:13:32.644922 26765 ts_tablet_manager.cc:1939] T 8255be28fb544d898425f886cb03fbaf P 539a904adf304706ba4bb28d27da130d: Deleting consensus metadata
I20260812 03:13:32.644922 26764 ts_tablet_manager.cc:1939] T 8255be28fb544d898425f886cb03fbaf P 875290aa5dea40a1a6370074aa72e6b2: Deleting consensus metadata
I20260812 03:13:32.646468 25865 catalog_manager.cc:4841] TS 539a904adf304706ba4bb28d27da130d (127.16.217.67:46457): tablet 8255be28fb544d898425f886cb03fbaf (table test-table [id=60505a2fa90141ea89f2cf97918fe3d3]) successfully deleted
I20260812 03:13:32.646827 25867 catalog_manager.cc:4841] TS 875290aa5dea40a1a6370074aa72e6b2 (127.16.217.66:41307): tablet 8255be28fb544d898425f886cb03fbaf (table test-table [id=60505a2fa90141ea89f2cf97918fe3d3]) successfully deleted
W20260812 03:13:32.652786 25867 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.217.65:33973 (error 108)
I20260812 03:13:32.653817 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 26226
I20260812 03:13:32.654268 26707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.653676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:33973 (local address 127.0.0.1:36166)
0812 03:13:32.654037 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:32.654130 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260812 03:13:32.664260 25867 catalog_manager.cc:4584] TS 56685a979f90401bba308f0e40ec13e3 (127.16.217.65:33973): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8255be28fb544d898425f886cb03fbaf: Network error: Client connection negotiation failed: client connection to 127.16.217.65:33973: connect: Connection refused (error 111)
I20260812 03:13:32.670423 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 26466
I20260812 03:13:32.686024 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 25826
2026-08-12T03:13:32Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1144 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 03:13:32.713228 17253 mini_oidc.cc:152] Starting JWKS server
I20260812 03:13:32.715332 17253 webserver.cc:460] Webserver started at https://127.0.0.1:34829/ using document root <none> and password file <none>
I20260812 03:13:32.715533 17253 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 03:13:32.715864 17253 webserver.cc:460] Webserver started at http://127.0.0.1:44095/ using document root <none> and password file <none>
2026-08-12T03:13:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:32Z Disabled control of system clock
I20260812 03:13:32.727223 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:33579
--webserver_interface=127.16.217.126
--webserver_port=0
--builtin_ntp_servers=127.16.217.84:37087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:33579
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34829/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/testchainca.cert with env {}
W20260812 03:13:32.842797 26780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:32.843072 26780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:32.843114 26780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:32.847277 26780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:32.847345 26780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:32.847361 26780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:32.847378 26780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:32.847393 26780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:13:32.847462 26780 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:13:32.847491 26780 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34829/jwks.json
I20260812 03:13:32.852926 26780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:33579
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:33579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34829/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:32.854084 26780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:32.855264 26780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:32.864565 26785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:32.864770 26780 server_base.cc:1034] running on GCE node
W20260812 03:13:32.864581 26787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:32.865001 26789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:32.865615 26780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:32.866891 26780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:32.868080 26780 hybrid_clock.cc:648] HybridClock initialized: now 1786504412868042 us; error 44 us; skew 500 ppm
I20260812 03:13:32.870044 26780 webserver.cc:460] Webserver started at http://127.16.217.126:44173/ using document root <none> and password file <none>
I20260812 03:13:32.870785 26780 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:32.870851 26780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:32.871083 26780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:32.873188 26780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "9a40d0de3f3b404cac883fd377a94f0d"
format_stamp: "Formatted at 2026-08-12 03:13:32 on dist-test-slave-7nm7"
server_key: "739bd028cf2a23904f9cada8bc936a48"
server_key_iv: "14022fd0a33aa29286e0b5244295a4d4"
server_key_version: "encryptionkey@0"
I20260812 03:13:32.873776 26780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "9a40d0de3f3b404cac883fd377a94f0d"
format_stamp: "Formatted at 2026-08-12 03:13:32 on dist-test-slave-7nm7"
server_key: "739bd028cf2a23904f9cada8bc936a48"
server_key_iv: "14022fd0a33aa29286e0b5244295a4d4"
server_key_version: "encryptionkey@0"
I20260812 03:13:32.877707 26780 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:13:32.880297 26799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:32.881588 26780 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:13:32.881747 26780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "9a40d0de3f3b404cac883fd377a94f0d"
format_stamp: "Formatted at 2026-08-12 03:13:32 on dist-test-slave-7nm7"
server_key: "739bd028cf2a23904f9cada8bc936a48"
server_key_iv: "14022fd0a33aa29286e0b5244295a4d4"
server_key_version: "encryptionkey@0"
I20260812 03:13:32.881868 26780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:32.903569 26780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:32.904390 26780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:32.904601 26780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:32.912408 26780 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:33579
I20260812 03:13:32.912415 26891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:33579 every 8 connection(s)
I20260812 03:13:32.913446 26780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:32.914600 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 26780
I20260812 03:13:32.914759 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:13:32.915094 17253 external_mini_cluster.cc:1428] Setting key 59b1fa02e50009ba65b6878296b94062
I20260812 03:13:32.917825 26892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:32.924029 26892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d: Bootstrap starting.
I20260812 03:13:32.925020 26897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:32.916637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54978 (local address 127.16.217.126:33579)
0812 03:13:32.917401 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:32.917419 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:32.917457 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:13:32.918188 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:32.918207 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:32.918674 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:32.919011 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:32.919441 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:32.920412 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:32.921326 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:32.921802 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:32.921976 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:32.922919 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:32.922924 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:32.922937 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:13:32.923327 (+ 390us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:32.923383 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:13:32.923403 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:32.923511 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:32.923691 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 03:13:32.924006 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":154,"threads_started":1}
I20260812 03:13:32.926692 26892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:32.927834 26892 log.cc:824] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:32.928680 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:13:32.930346 26892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d: No bootstrap required, opened a new log
I20260812 03:13:32.934316 26892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a40d0de3f3b404cac883fd377a94f0d" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 33579 } }
I20260812 03:13:32.934664 26892 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:32.934733 26892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a40d0de3f3b404cac883fd377a94f0d, State: Initialized, Role: FOLLOWER
I20260812 03:13:32.935169 26892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a40d0de3f3b404cac883fd377a94f0d" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 33579 } }
I20260812 03:13:32.935297 26892 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:32.935339 26892 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:32.935412 26892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:32.936443 26892 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a40d0de3f3b404cac883fd377a94f0d" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 33579 } }
I20260812 03:13:32.936770 26892 leader_election.cc:302] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a40d0de3f3b404cac883fd377a94f0d; no voters:
I20260812 03:13:32.937150 26892 leader_election.cc:288] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:13:32.937286 26900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:32.937548 26900 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d [term 1 LEADER]: Becoming Leader. State: Replica: 9a40d0de3f3b404cac883fd377a94f0d, State: Running, Role: LEADER
I20260812 03:13:32.937891 26900 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a40d0de3f3b404cac883fd377a94f0d" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 33579 } }
I20260812 03:13:32.938575 26892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:32.939839 26902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a40d0de3f3b404cac883fd377a94f0d. Latest consensus state: current_term: 1 leader_uuid: "9a40d0de3f3b404cac883fd377a94f0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a40d0de3f3b404cac883fd377a94f0d" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 33579 } } }
I20260812 03:13:32.940007 26902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:32.939845 26901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a40d0de3f3b404cac883fd377a94f0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a40d0de3f3b404cac883fd377a94f0d" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 33579 } } }
I20260812 03:13:32.940260 26901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:32.940541 26917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:32.945037 26917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:32.951571 26917 catalog_manager.cc:1347] Generated new cluster ID: 4bb744b3a77c4fbeb6087cdc7d73e65a
I20260812 03:13:32.951678 26917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:32.979146 26917 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:13:32.980808 26917 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:32.992635 26917 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a40d0de3f3b404cac883fd377a94f0d: Generated new TSK 0
I20260812 03:13:32.993394 26917 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 03:13:33.026818 26949 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 03:13:33.027046 26949 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 03:13:33.027554 26897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:33.006567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54992 (local address 127.16.217.126:33579)
0812 03:13:33.006749 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:33.006755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:33.006936 (+ 181us) server_negotiation.cc:407] Connection header received
0812 03:13:33.007153 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:33.007156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:33.007226 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:33.007347 (+ 121us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:13:33.007660 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:33.008498 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:33.009480 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:33.009839 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:33.009949 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:33.010458 (+ 509us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:13:33.027102 (+ 16644us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34829/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 03:13:33.027395 (+ 293us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54992: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34829/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 03:13:33.027621 26897 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54992: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34829/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 03:13:33.029055 26948 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:33.006486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:33579 (local address 127.0.0.1:54992)
0812 03:13:33.006845 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:33.006859 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:33.006982 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:33.007405 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:33.007407 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:33.007424 (+ 17us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:13:33.007533 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:33.007538 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:33.008687 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:33.008690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:33.009360 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:33.009367 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:33.010016 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:33.010019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:33.010104 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:33.010161 (+ 57us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:13:33.027543 (+ 17382us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34829/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 03:13:33.027839 (+ 296us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.217.126:33579: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34829/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":130,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.16.217.126:33579: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34829/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 03:13:33.029985 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 26780
2026-08-12T03:13:33Z chronyd exiting
I20260812 03:13:33.078583 17253 test_util.cc:175] -----------------------------------------------
I20260812 03:13:33.078675 17253 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504382969880-17253-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (367 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 03:13:33.079788 17253 mini_oidc.cc:152] Starting JWKS server
I20260812 03:13:33.080621 17253 webserver.cc:460] Webserver started at https://127.0.0.1:38399/ using document root <none> and password file <none>
I20260812 03:13:33.080704 17253 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 03:13:33.080904 17253 webserver.cc:460] Webserver started at http://127.0.0.1:36831/ using document root <none> and password file <none>
2026-08-12T03:13:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:33Z Disabled control of system clock
I20260812 03:13:33.092693 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:43895
--webserver_interface=127.16.217.126
--webserver_port=0
--builtin_ntp_servers=127.16.217.84:33779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:43895
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38399/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/testchainca.cert with env {}
W20260812 03:13:33.209507 26958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:33.209761 26958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:33.209800 26958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:33.213711 26958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:33.213765 26958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:33.213781 26958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:33.213797 26958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:33.213814 26958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:13:33.213829 26958 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:13:33.213842 26958 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38399/jwks.json
I20260812 03:13:33.219115 26958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:33779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:43895
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:43895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38399/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:33.220265 26958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:33.221356 26958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:33.230671 26965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:33.230702 26964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:33.230671 26969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:33.230960 26958 server_base.cc:1034] running on GCE node
I20260812 03:13:33.231599 26958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:33.232851 26958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:33.234069 26958 hybrid_clock.cc:648] HybridClock initialized: now 1786504413234025 us; error 61 us; skew 500 ppm
I20260812 03:13:33.236143 26958 webserver.cc:460] Webserver started at http://127.16.217.126:37701/ using document root <none> and password file <none>
I20260812 03:13:33.236783 26958 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:33.236886 26958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:33.237131 26958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:33.239317 26958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "eab7657b67bf4e64ba1e6ae2e0eef7cf"
format_stamp: "Formatted at 2026-08-12 03:13:33 on dist-test-slave-7nm7"
server_key: "c3f07336e9eeba0226163952b400de84"
server_key_iv: "a4bb208abe226a6a815b85ca4003bf26"
server_key_version: "encryptionkey@0"
I20260812 03:13:33.239925 26958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "eab7657b67bf4e64ba1e6ae2e0eef7cf"
format_stamp: "Formatted at 2026-08-12 03:13:33 on dist-test-slave-7nm7"
server_key: "c3f07336e9eeba0226163952b400de84"
server_key_iv: "a4bb208abe226a6a815b85ca4003bf26"
server_key_version: "encryptionkey@0"
I20260812 03:13:33.243754 26958 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:13:33.246392 26983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:33.247717 26958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:33.247838 26958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "eab7657b67bf4e64ba1e6ae2e0eef7cf"
format_stamp: "Formatted at 2026-08-12 03:13:33 on dist-test-slave-7nm7"
server_key: "c3f07336e9eeba0226163952b400de84"
server_key_iv: "a4bb208abe226a6a815b85ca4003bf26"
server_key_version: "encryptionkey@0"
I20260812 03:13:33.247932 26958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:33.261236 26958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:33.261943 26958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:33.262115 26958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:33.270319 26958 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:43895
I20260812 03:13:33.270414 27078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:43895 every 8 connection(s)
I20260812 03:13:33.271409 26958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:33.274814 27079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:33.280519 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 26958
I20260812 03:13:33.280640 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:13:33.280978 17253 external_mini_cluster.cc:1428] Setting key e9da591cc3c490280c3c13789e2af4ae
I20260812 03:13:33.283926 27079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf: Bootstrap starting.
I20260812 03:13:33.287456 27079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:33.288329 27079 log.cc:824] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:33.291009 27079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf: No bootstrap required, opened a new log
I20260812 03:13:33.291822 27084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:33.282849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57096 (local address 127.16.217.126:43895)
0812 03:13:33.283572 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:33.283587 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:33.283624 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:13:33.284486 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:33.284506 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:33.284937 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:33.285279 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:33.285740 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:33.286835 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:33.287760 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:33.288249 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:33.288425 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:33.289441 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:33.289446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:33.289460 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:13:33.290019 (+ 559us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:33.290113 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:13:33.290140 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:33.290305 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:33.290745 (+ 440us) server_negotiation.cc:299] Negotiation successful
0812 03:13:33.291004 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":171,"threads_started":1}
I20260812 03:13:33.295161 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:13:33.295473 27079 raft_consensus.cc:348] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab7657b67bf4e64ba1e6ae2e0eef7cf" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43895 } }
I20260812 03:13:33.295733 27079 raft_consensus.cc:374] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:33.295812 27079 raft_consensus.cc:729] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eab7657b67bf4e64ba1e6ae2e0eef7cf, State: Initialized, Role: FOLLOWER
I20260812 03:13:33.296335 27079 consensus_queue.cc:260] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab7657b67bf4e64ba1e6ae2e0eef7cf" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43895 } }
I20260812 03:13:33.296522 27079 raft_consensus.cc:388] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:33.296622 27079 raft_consensus.cc:482] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:33.296729 27079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:33.298741 27079 raft_consensus.cc:504] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab7657b67bf4e64ba1e6ae2e0eef7cf" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43895 } }
I20260812 03:13:33.299475 27079 leader_election.cc:302] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eab7657b67bf4e64ba1e6ae2e0eef7cf; no voters:
I20260812 03:13:33.300171 27079 leader_election.cc:288] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:13:33.300201 27086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:33.300721 27086 raft_consensus.cc:686] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf [term 1 LEADER]: Becoming Leader. State: Replica: eab7657b67bf4e64ba1e6ae2e0eef7cf, State: Running, Role: LEADER
I20260812 03:13:33.301429 27086 consensus_queue.cc:237] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab7657b67bf4e64ba1e6ae2e0eef7cf" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43895 } }
I20260812 03:13:33.302584 27079 sys_catalog.cc:564] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:33.303493 27093 sys_catalog.cc:455] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader eab7657b67bf4e64ba1e6ae2e0eef7cf. Latest consensus state: current_term: 1 leader_uuid: "eab7657b67bf4e64ba1e6ae2e0eef7cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab7657b67bf4e64ba1e6ae2e0eef7cf" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43895 } } }
I20260812 03:13:33.303640 27093 sys_catalog.cc:458] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:33.303478 27087 sys_catalog.cc:455] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eab7657b67bf4e64ba1e6ae2e0eef7cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab7657b67bf4e64ba1e6ae2e0eef7cf" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43895 } } }
I20260812 03:13:33.303942 27087 sys_catalog.cc:458] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:33.304217 27102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:33.307708 27102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:33.315474 27102 catalog_manager.cc:1347] Generated new cluster ID: 35b5ae4525f544e0b8fa8240a8b950be
I20260812 03:13:33.315619 27102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:33.327615 27102 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:13:33.328675 27102 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:33.339833 27102 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eab7657b67bf4e64ba1e6ae2e0eef7cf: Generated new TSK 0
I20260812 03:13:33.340626 27102 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 03:13:33.374398 27123 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 03:13:33.374507 27123 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 03:13:33.375368 27084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:33.350467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57102 (local address 127.16.217.126:43895)
0812 03:13:33.350620 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:33.350625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:33.350766 (+ 141us) server_negotiation.cc:407] Connection header received
0812 03:13:33.350978 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:33.350980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:33.351031 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:33.351155 (+ 124us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:13:33.351404 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:33.352081 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:33.352982 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:33.353236 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:33.353351 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:33.353683 (+ 332us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:13:33.374856 (+ 21173us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38399/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 03:13:33.375195 (+ 339us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57102: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38399/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 03:13:33.375452 27084 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57102: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38399/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 03:13:33.375478 27122 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:33.350343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43895 (local address 127.0.0.1:57102)
0812 03:13:33.350654 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:33.350668 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:33.350820 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:33.351152 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:33.351155 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:33.351163 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:13:33.351282 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:33.351288 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:33.352199 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:33.352205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:33.352842 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:33.352850 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:33.353368 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:33.353372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:33.353461 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:33.353494 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:13:33.375102 (+ 21608us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38399/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 03:13:33.375289 (+ 187us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.217.126:43895: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38399/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":108,"threads_started":1}
I20260812 03:13:33.376240 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 26958
2026-08-12T03:13:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (352 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 03:13:33.432116 17253 mini_oidc.cc:152] Starting JWKS server
I20260812 03:13:33.432974 17253 webserver.cc:460] Webserver started at https://127.0.0.1:41247/ using document root <none> and password file <none>
I20260812 03:13:33.433053 17253 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 03:13:33.433319 17253 webserver.cc:460] Webserver started at http://127.0.0.1:33823/ using document root <none> and password file <none>
2026-08-12T03:13:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:33Z Disabled control of system clock
I20260812 03:13:33.444573 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:32821
--webserver_interface=127.16.217.126
--webserver_port=0
--builtin_ntp_servers=127.16.217.84:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:32821
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41247/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 03:13:33.572435 27137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:33.572858 27137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:33.572934 27137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:33.581063 27137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:33.581173 27137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:33.581204 27137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:33.581238 27137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:33.581276 27137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:13:33.581316 27137 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:13:33.581342 27137 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41247/jwks.json
I20260812 03:13:33.591526 27137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:32821
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:32821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41247/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:33.593235 27137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:33.595328 27137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:33.603942 27152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:33.603942 27145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:33.603971 27154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:33.604301 27137 server_base.cc:1034] running on GCE node
I20260812 03:13:33.604873 27137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:33.605887 27137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:33.607079 27137 hybrid_clock.cc:648] HybridClock initialized: now 1786504413607052 us; error 39 us; skew 500 ppm
I20260812 03:13:33.608911 27137 webserver.cc:460] Webserver started at http://127.16.217.126:39349/ using document root <none> and password file <none>
I20260812 03:13:33.609510 27137 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:33.609604 27137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:33.609850 27137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:33.611979 27137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "c4b9a2ae917748c5a6d5135805a4d1e9"
format_stamp: "Formatted at 2026-08-12 03:13:33 on dist-test-slave-7nm7"
server_key: "0a008c19b07e6adb4786e33694ff63ff"
server_key_iv: "d4bbe7937cb11e713931dea294fa4feb"
server_key_version: "encryptionkey@0"
I20260812 03:13:33.612541 27137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "c4b9a2ae917748c5a6d5135805a4d1e9"
format_stamp: "Formatted at 2026-08-12 03:13:33 on dist-test-slave-7nm7"
server_key: "0a008c19b07e6adb4786e33694ff63ff"
server_key_iv: "d4bbe7937cb11e713931dea294fa4feb"
server_key_version: "encryptionkey@0"
I20260812 03:13:33.616243 27137 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260812 03:13:33.618924 27162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:33.620250 27137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:33.620414 27137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "c4b9a2ae917748c5a6d5135805a4d1e9"
format_stamp: "Formatted at 2026-08-12 03:13:33 on dist-test-slave-7nm7"
server_key: "0a008c19b07e6adb4786e33694ff63ff"
server_key_iv: "d4bbe7937cb11e713931dea294fa4feb"
server_key_version: "encryptionkey@0"
I20260812 03:13:33.620543 27137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:33.632754 27137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:33.633443 27137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:33.633667 27137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:33.642872 27137 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:32821
I20260812 03:13:33.642879 27255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:32821 every 8 connection(s)
I20260812 03:13:33.643884 27137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:33.645031 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 27137
I20260812 03:13:33.645179 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:13:33.645673 17253 external_mini_cluster.cc:1428] Setting key 202aa6339a5440f16dacc91cbed549d5
I20260812 03:13:33.648733 27256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:33.655536 27259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:33.647292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40230 (local address 127.16.217.126:32821)
0812 03:13:33.647979 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:33.647995 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:33.648031 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:13:33.648873 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:33.648899 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:33.649312 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:33.649603 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:33.649942 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:33.650999 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:33.652234 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:33.652694 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:33.652817 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:33.653766 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:33.653771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:33.653785 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:13:33.654197 (+ 412us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:33.654315 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:13:33.654339 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:33.654449 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:33.654564 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 03:13:33.654823 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":177,"threads_started":1}
I20260812 03:13:33.656081 27256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9: Bootstrap starting.
I20260812 03:13:33.658869 27256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:33.659060 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:13:33.659797 27256 log.cc:824] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:33.661985 27256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9: No bootstrap required, opened a new log
I20260812 03:13:33.665683 27256 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b9a2ae917748c5a6d5135805a4d1e9" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 32821 } }
I20260812 03:13:33.665908 27256 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:33.665994 27256 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b9a2ae917748c5a6d5135805a4d1e9, State: Initialized, Role: FOLLOWER
I20260812 03:13:33.666493 27256 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b9a2ae917748c5a6d5135805a4d1e9" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 32821 } }
I20260812 03:13:33.666626 27256 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:33.666667 27256 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:33.666750 27256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:33.667645 27256 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b9a2ae917748c5a6d5135805a4d1e9" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 32821 } }
I20260812 03:13:33.667956 27256 leader_election.cc:302] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4b9a2ae917748c5a6d5135805a4d1e9; no voters:
I20260812 03:13:33.668255 27256 leader_election.cc:288] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:13:33.668399 27263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:33.668733 27263 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9 [term 1 LEADER]: Becoming Leader. State: Replica: c4b9a2ae917748c5a6d5135805a4d1e9, State: Running, Role: LEADER
I20260812 03:13:33.669124 27263 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b9a2ae917748c5a6d5135805a4d1e9" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 32821 } }
I20260812 03:13:33.669554 27256 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:33.671346 27270 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4b9a2ae917748c5a6d5135805a4d1e9. Latest consensus state: current_term: 1 leader_uuid: "c4b9a2ae917748c5a6d5135805a4d1e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b9a2ae917748c5a6d5135805a4d1e9" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 32821 } } }
I20260812 03:13:33.671520 27270 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:33.671346 27264 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4b9a2ae917748c5a6d5135805a4d1e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b9a2ae917748c5a6d5135805a4d1e9" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 32821 } } }
I20260812 03:13:33.671852 27264 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:33.671903 27286 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:33.675127 27286 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:33.682061 27286 catalog_manager.cc:1347] Generated new cluster ID: 66d1c4f5c7d94c0996b18aa7b2c1908d
I20260812 03:13:33.682153 27286 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:33.701261 27286 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:13:33.702767 27286 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:33.709939 27286 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4b9a2ae917748c5a6d5135805a4d1e9: Generated new TSK 0
I20260812 03:13:33.710870 27286 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:13:33.726706 27259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:33.721479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40244 (local address 127.16.217.126:32821)
0812 03:13:33.721627 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:33.721631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:33.721864 (+ 233us) server_negotiation.cc:407] Connection header received
0812 03:13:33.722075 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:33.722078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:33.722129 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:33.722271 (+ 142us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:13:33.722516 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:33.723061 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:33.724061 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:33.724278 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:33.724388 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:33.724815 (+ 427us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:13:33.726343 (+ 1528us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41247/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 03:13:33.726564 (+ 221us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40244: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41247/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 03:13:33.726783 27259 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40244: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41247/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 03:13:33.728484 27304 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:33.721406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:32821 (local address 127.0.0.1:40244)
0812 03:13:33.721773 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:33.721784 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:33.721915 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:33.722270 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:33.722274 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:33.722282 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:13:33.722391 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:33.722399 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:33.723186 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:33.723193 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:33.723912 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:33.723920 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:33.724413 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:33.724418 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:33.724510 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:33.724533 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:13:33.726539 (+ 2006us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41247/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 03:13:33.728297 (+ 1758us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.217.126:32821: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41247/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":1534,"thread_start_us":108,"threads_started":1}
I20260812 03:13:33.729259 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 27137
E20260812 03:13:33.744177 27305 webserver.cc:533] Webserver: sslize failed:
E20260812 03:13:33.744287 27305 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T03:13:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (344 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T03:13:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:33Z Disabled control of system clock
I20260812 03:13:33.788672 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:39113
--webserver_interface=127.16.217.126
--webserver_port=0
--builtin_ntp_servers=127.16.217.84:37163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:39113
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/testchainca.cert with env {}
W20260812 03:13:33.911051 27315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:33.911319 27315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:33.911360 27315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:33.915333 27315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:33.915395 27315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:33.915416 27315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:33.915433 27315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:33.915448 27315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:13:33.915465 27315 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:13:33.915478 27315 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 03:13:33.920804 27315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:39113
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:39113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:33.922642 27315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:33.923648 27315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:33.933323 27323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:33.933418 27322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:33.933348 27326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:33.933470 27315 server_base.cc:1034] running on GCE node
I20260812 03:13:33.934293 27315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:33.935412 27315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:33.936600 27315 hybrid_clock.cc:648] HybridClock initialized: now 1786504413936575 us; error 39 us; skew 500 ppm
I20260812 03:13:33.938431 27315 webserver.cc:460] Webserver started at http://127.16.217.126:46805/ using document root <none> and password file <none>
I20260812 03:13:33.939010 27315 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:33.939105 27315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:33.939311 27315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:33.941316 27315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "9bf27f22097344c0882b542580fd1326"
format_stamp: "Formatted at 2026-08-12 03:13:33 on dist-test-slave-7nm7"
server_key: "f807512bf9df4c4de7dbac0e06a9abe1"
server_key_iv: "32e2b0b5b927af50bf47ce2dbec661ad"
server_key_version: "encryptionkey@0"
I20260812 03:13:33.941881 27315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "9bf27f22097344c0882b542580fd1326"
format_stamp: "Formatted at 2026-08-12 03:13:33 on dist-test-slave-7nm7"
server_key: "f807512bf9df4c4de7dbac0e06a9abe1"
server_key_iv: "32e2b0b5b927af50bf47ce2dbec661ad"
server_key_version: "encryptionkey@0"
I20260812 03:13:33.945669 27315 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:13:33.948134 27336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:33.949707 27315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:33.949836 27315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "9bf27f22097344c0882b542580fd1326"
format_stamp: "Formatted at 2026-08-12 03:13:33 on dist-test-slave-7nm7"
server_key: "f807512bf9df4c4de7dbac0e06a9abe1"
server_key_iv: "32e2b0b5b927af50bf47ce2dbec661ad"
server_key_version: "encryptionkey@0"
I20260812 03:13:33.949946 27315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:33.973989 27315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:33.974792 27315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:33.974999 27315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:33.982492 27315 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:39113
I20260812 03:13:33.982512 27423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:39113 every 8 connection(s)
I20260812 03:13:33.983436 27315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:33.985708 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 27315
I20260812 03:13:33.985810 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:13:33.986135 17253 external_mini_cluster.cc:1428] Setting key d22d7b01d3f56667cdf186242c8381cb
I20260812 03:13:33.986481 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:33.994165 27425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326: Bootstrap starting.
I20260812 03:13:33.996032 27428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:33.987657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48146 (local address 127.16.217.126:39113)
0812 03:13:33.988357 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:33.988373 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:33.988409 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:13:33.989319 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:33.989339 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:33.989731 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:33.990094 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:33.990548 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:33.991508 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:33.992395 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:33.992863 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:33.993034 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:33.994177 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:33.994182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:33.994195 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:13:33.994701 (+ 506us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:33.994757 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:13:33.994779 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:33.994892 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:33.995044 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 03:13:33.995311 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":153,"threads_started":1}
I20260812 03:13:33.996829 27425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:33.997615 27425 log.cc:824] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:33.999874 27425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326: No bootstrap required, opened a new log
I20260812 03:13:33.999869 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:13:34.003672 27425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf27f22097344c0882b542580fd1326" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39113 } }
I20260812 03:13:34.003891 27425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:34.003952 27425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bf27f22097344c0882b542580fd1326, State: Initialized, Role: FOLLOWER
I20260812 03:13:34.004365 27425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf27f22097344c0882b542580fd1326" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39113 } }
I20260812 03:13:34.004487 27425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:34.004554 27425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:34.004675 27425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:34.005651 27425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf27f22097344c0882b542580fd1326" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39113 } }
I20260812 03:13:34.006011 27425 leader_election.cc:302] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bf27f22097344c0882b542580fd1326; no voters:
I20260812 03:13:34.006368 27425 leader_election.cc:288] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:13:34.006493 27441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:34.006806 27441 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326 [term 1 LEADER]: Becoming Leader. State: Replica: 9bf27f22097344c0882b542580fd1326, State: Running, Role: LEADER
I20260812 03:13:34.007130 27441 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf27f22097344c0882b542580fd1326" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39113 } }
I20260812 03:13:34.007707 27425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:34.008802 27443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bf27f22097344c0882b542580fd1326. Latest consensus state: current_term: 1 leader_uuid: "9bf27f22097344c0882b542580fd1326" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf27f22097344c0882b542580fd1326" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39113 } } }
I20260812 03:13:34.008950 27443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:34.009307 27445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bf27f22097344c0882b542580fd1326" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf27f22097344c0882b542580fd1326" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39113 } } }
I20260812 03:13:34.009399 27445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:34.009384 27453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:34.012741 27453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:34.017982 27453 catalog_manager.cc:1347] Generated new cluster ID: 82edcc6ddbcf4f2a8c3409ceb6ab9101
I20260812 03:13:34.018049 27453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:34.035303 27453 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:13:34.036240 27453 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:34.051401 27453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bf27f22097344c0882b542580fd1326: Generated new TSK 0
I20260812 03:13:34.052217 27453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:13:34.135681 27428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:34.062482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48156 (local address 127.16.217.126:39113)
0812 03:13:34.062616 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:34.062621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:34.062740 (+ 119us) server_negotiation.cc:407] Connection header received
0812 03:13:34.062949 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:34.062954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:34.063011 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:34.063139 (+ 128us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:13:34.063382 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:34.063937 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:34.064845 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:34.065097 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:34.065210 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:34.065556 (+ 346us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:13:34.135149 (+ 69593us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 03:13:34.135494 (+ 345us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48156: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 03:13:34.135780 27428 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48156: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 03:13:34.135798 27474 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:34.062312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39113 (local address 127.0.0.1:48156)
0812 03:13:34.062634 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:34.062650 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:34.062805 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:34.063136 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:34.063140 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:34.063148 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:13:34.063263 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:34.063277 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:34.064058 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:34.064062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:34.064702 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:34.064722 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:34.065235 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:34.065239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:34.065338 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:34.065361 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:13:34.135401 (+ 70040us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 03:13:34.135603 (+ 202us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.217.126:39113: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":122,"threads_started":1}
I20260812 03:13:34.136626 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 27315
2026-08-12T03:13:34Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (388 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (129 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (129 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:13:34 dist-test-slave-7nm7 krb5kdc[27492](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:13:34 dist-test-slave-7nm7 krb5kdc[27492](info): set up 2 sockets
Aug 12 03:13:34 dist-test-slave-7nm7 krb5kdc[27492](info): commencing operation
krb5kdc: starting...
W20260812 03:13:35.749454 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.325s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:13:35 dist-test-slave-7nm7 krb5kdc[27492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:13:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:13:35.881215 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:38625
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:38625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:35.995924 27518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:35.996222 27518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:35.996265 27518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:36.000447 27518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:36.000525 27518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:36.000545 27518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:36.000566 27518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:36.000581 27518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:13:36.006475 27518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:38625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:38625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:36.008504 27518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:36.009902 27518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:36.019028 27529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:36.018947 27525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:36.019141 27518 server_base.cc:1034] running on GCE node
W20260812 03:13:36.019032 27526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:36.019819 27518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:36.020929 27518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:36.022220 27518 hybrid_clock.cc:648] HybridClock initialized: now 1786504416022166 us; error 44 us; skew 500 ppm
Aug 12 03:13:36 dist-test-slave-7nm7 krb5kdc[27492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504416, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:36.031795 27518 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:13:36.032938 27518 webserver.cc:460] Webserver started at http://127.16.217.126:41405/ using document root <none> and password file <none>
I20260812 03:13:36.033587 27518 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:36.033659 27518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:36.033896 27518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:36.036233 27518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "cde2ce6dcf084855b7bed44336f7d641"
format_stamp: "Formatted at 2026-08-12 03:13:36 on dist-test-slave-7nm7"
server_key: "c2418ecdd8398e49f9a8cb9a69ffe980"
server_key_iv: "29331b9cf2640c586dd39d82000371e3"
server_key_version: "encryptionkey@0"
I20260812 03:13:36.036885 27518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "cde2ce6dcf084855b7bed44336f7d641"
format_stamp: "Formatted at 2026-08-12 03:13:36 on dist-test-slave-7nm7"
server_key: "c2418ecdd8398e49f9a8cb9a69ffe980"
server_key_iv: "29331b9cf2640c586dd39d82000371e3"
server_key_version: "encryptionkey@0"
I20260812 03:13:36.041603 27518 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:13:36.044816 27540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:36.046187 27518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:36.046384 27518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "cde2ce6dcf084855b7bed44336f7d641"
format_stamp: "Formatted at 2026-08-12 03:13:36 on dist-test-slave-7nm7"
server_key: "c2418ecdd8398e49f9a8cb9a69ffe980"
server_key_iv: "29331b9cf2640c586dd39d82000371e3"
server_key_version: "encryptionkey@0"
I20260812 03:13:36.046514 27518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:36.067896 27518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:36.071261 27518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:36.071519 27518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:36.079495 27518 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:38625
I20260812 03:13:36.079507 27634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:38625 every 8 connection(s)
I20260812 03:13:36.080538 27518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:36.083559 27635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:36.088788 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 27518
I20260812 03:13:36.088922 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:13:36.089203 17253 external_mini_cluster.cc:1428] Setting key e86ba4e7f213a463d382e1b043d5c3aa
I20260812 03:13:36.089563 27635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641: Bootstrap starting.
I20260812 03:13:36.092089 27635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:36.093061 27635 log.cc:824] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:36.095299 27635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641: No bootstrap required, opened a new log
Aug 12 03:13:36 dist-test-slave-7nm7 krb5kdc[27492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:36.100301 27635 raft_consensus.cc:348] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde2ce6dcf084855b7bed44336f7d641" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 38625 } }
I20260812 03:13:36.100582 27635 raft_consensus.cc:374] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:36.100673 27635 raft_consensus.cc:729] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cde2ce6dcf084855b7bed44336f7d641, State: Initialized, Role: FOLLOWER
I20260812 03:13:36.101202 27635 consensus_queue.cc:260] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde2ce6dcf084855b7bed44336f7d641" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 38625 } }
I20260812 03:13:36.101368 27635 raft_consensus.cc:388] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:36.101464 27635 raft_consensus.cc:482] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:36.101572 27635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:36.102645 27635 raft_consensus.cc:504] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde2ce6dcf084855b7bed44336f7d641" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 38625 } }
I20260812 03:13:36.103008 27635 leader_election.cc:302] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cde2ce6dcf084855b7bed44336f7d641; no voters:
I20260812 03:13:36.103430 27635 leader_election.cc:288] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:13:36.103514 27640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:36.103763 27640 raft_consensus.cc:686] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641 [term 1 LEADER]: Becoming Leader. State: Replica: cde2ce6dcf084855b7bed44336f7d641, State: Running, Role: LEADER
I20260812 03:13:36.104086 27640 consensus_queue.cc:237] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde2ce6dcf084855b7bed44336f7d641" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 38625 } }
I20260812 03:13:36.104472 27638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:36.090661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37654 (local address 127.16.217.126:38625)
0812 03:13:36.091089 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:36.091106 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:36.091142 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:13:36.091776 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:36.091793 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:36.092121 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:36.092450 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:36.093349 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.094377 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.095519 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.095988 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.096115 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:36.098962 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:36.098967 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:36.098980 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:36.099007 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:36.101211 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:36.101748 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:36.101761 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:36.101765 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:36.101876 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:36.102174 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:36.102178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:36.102180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:36.102651 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:36.102854 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:36.103125 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 03:13:36.103453 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":140,"threads_started":1}
I20260812 03:13:36.104992 27635 sys_catalog.cc:564] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:36.105324 27640 sys_catalog.cc:455] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cde2ce6dcf084855b7bed44336f7d641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde2ce6dcf084855b7bed44336f7d641" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 38625 } } }
I20260812 03:13:36.105443 27640 sys_catalog.cc:458] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:36.105865 27641 sys_catalog.cc:455] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cde2ce6dcf084855b7bed44336f7d641. Latest consensus state: current_term: 1 leader_uuid: "cde2ce6dcf084855b7bed44336f7d641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde2ce6dcf084855b7bed44336f7d641" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 38625 } } }
I20260812 03:13:36.106048 27641 sys_catalog.cc:458] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:36.106925 27647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:36.111300 27647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:36.116770 27647 catalog_manager.cc:1347] Generated new cluster ID: 692294c5185c402ab1b253f86d667701
I20260812 03:13:36.116832 27647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:36.136636 27647 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:13:36.137609 27647 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:36.145154 27647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cde2ce6dcf084855b7bed44336f7d641: Generated new TSK 0
I20260812 03:13:36.145920 27647 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
I20260812 03:13:36.183197 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:38625
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:36.320007 27681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:36.320282 27681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:36.320327 27681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:36.324532 27681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:36.324592 27681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:36.324702 27681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:13:36.329968 27681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:38625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:36.331120 27681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:36.332182 27681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:36.342530 27698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:36.342592 27692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:36.342567 27693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:36.342739 27681 server_base.cc:1034] running on GCE node
I20260812 03:13:36.343351 27681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:36.344034 27681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:36.345244 27681 hybrid_clock.cc:648] HybridClock initialized: now 1786504416345231 us; error 25 us; skew 500 ppm
Aug 12 03:13:36 dist-test-slave-7nm7 krb5kdc[27492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504416, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:36.348304 27681 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:13:36.349239 27681 webserver.cc:460] Webserver started at http://127.16.217.65:34377/ using document root <none> and password file <none>
I20260812 03:13:36.349875 27681 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:36.349972 27681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:36.350448 27681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:36.352764 27681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "5e945b932f4b41cd84c6109519ac86d8"
format_stamp: "Formatted at 2026-08-12 03:13:36 on dist-test-slave-7nm7"
server_key: "08cc2e7c489af688a65777178c5415de"
server_key_iv: "5b360b1f566efb9583d720d250cfa008"
server_key_version: "encryptionkey@0"
I20260812 03:13:36.353351 27681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "5e945b932f4b41cd84c6109519ac86d8"
format_stamp: "Formatted at 2026-08-12 03:13:36 on dist-test-slave-7nm7"
server_key: "08cc2e7c489af688a65777178c5415de"
server_key_iv: "5b360b1f566efb9583d720d250cfa008"
server_key_version: "encryptionkey@0"
I20260812 03:13:36.357430 27681 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:13:36.360065 27710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:36.361325 27681 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:13:36.361469 27681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "5e945b932f4b41cd84c6109519ac86d8"
format_stamp: "Formatted at 2026-08-12 03:13:36 on dist-test-slave-7nm7"
server_key: "08cc2e7c489af688a65777178c5415de"
server_key_iv: "5b360b1f566efb9583d720d250cfa008"
server_key_version: "encryptionkey@0"
I20260812 03:13:36.361588 27681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:36.374382 27681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:36.377863 27681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:36.378088 27681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:36.378813 27681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:36.379700 27681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:36.379772 27681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:36.379846 27681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:36.379896 27681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:36.389696 27681 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:35147
I20260812 03:13:36.389735 27908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:35147 every 8 connection(s)
I20260812 03:13:36.390663 27681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:13:36.392759 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 27681
I20260812 03:13:36.392921 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:13:36.393289 17253 external_mini_cluster.cc:1428] Setting key 22e6045662b0dca28c7d5d3da67e3ff4
Aug 12 03:13:36 dist-test-slave-7nm7 krb5kdc[27492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504416, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.217.66@KRBTEST.COM; defaulting to no policy
I20260812 03:13:36.407878 27638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:36.393250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:46587 (local address 127.16.217.126:38625)
0812 03:13:36.393623 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:36.393627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:36.394538 (+ 911us) server_negotiation.cc:407] Connection header received
0812 03:13:36.396165 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:36.396170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:36.396239 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:36.396468 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:36.397740 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.398380 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.399693 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.400095 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.400223 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:36.403095 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:36.403103 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:36.403109 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:36.403146 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:36.405018 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:36.405618 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:36.405624 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:36.405627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:36.405719 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:36.406265 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:36.406272 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:36.406277 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:36.406518 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:36.406611 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:36.407453 (+ 842us) server_negotiation.cc:299] Negotiation successful
0812 03:13:36.407694 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 03:13:36.408574 27912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:36.393711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:38625 (local address 127.16.217.65:46587)
0812 03:13:36.394371 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:36.394430 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:36.395849 (+ 1419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:36.396671 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:36.396678 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:36.397065 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:36.397576 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:36.397600 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.398517 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.398521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:36.399544 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:36.399556 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.400287 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.400291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:36.400464 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:36.401232 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:36.401250 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:36.402901 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:36.405185 (+ 2284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:36.405191 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:36.405205 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:36.405499 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:36.405870 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:36.405876 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:36.405878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:36.406122 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:36.406635 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:36.406640 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:36.406896 (+ 256us) client_negotiation.cc:769] Sending connection context
0812 03:13:36.407223 (+ 327us) client_negotiation.cc:240] Negotiation successful
0812 03:13:36.407584 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":179,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.66@KRBTEST.COM" created.
I20260812 03:13:36.409936 27910 heartbeater.cc:347] Connected to a master server at 127.16.217.126:38625
I20260812 03:13:36.410192 27910 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:36.410857 27910 heartbeater.cc:511] Master 127.16.217.126:38625 requested a full tablet report, sending...
I20260812 03:13:36.412562 27571 ts_manager.cc:194] Registered new tserver with Master: 5e945b932f4b41cd84c6109519ac86d8 (127.16.217.65:35147)
I20260812 03:13:36.414316 27571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:46587
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
I20260812 03:13:36.429096 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:38625
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:36.556340 27918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:36.556769 27918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:36.556857 27918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:36.565141 27918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:36.565255 27918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:36.565361 27918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:13:36.572242 27918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:38625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:36.574054 27918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:36.575273 27918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:36.586082 27925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:36.586212 27928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:36.586297 27918 server_base.cc:1034] running on GCE node
W20260812 03:13:36.586165 27924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:36.586829 27918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:36.587491 27918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:36.588660 27918 hybrid_clock.cc:648] HybridClock initialized: now 1786504416588626 us; error 50 us; skew 500 ppm
Aug 12 03:13:36 dist-test-slave-7nm7 krb5kdc[27492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504416, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:36.591542 27918 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:13:36.592409 27918 webserver.cc:460] Webserver started at http://127.16.217.66:44477/ using document root <none> and password file <none>
I20260812 03:13:36.592944 27918 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:36.593004 27918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:36.593183 27918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:36.595309 27918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "64496d67263147bf9dc2005cd49b78a0"
format_stamp: "Formatted at 2026-08-12 03:13:36 on dist-test-slave-7nm7"
server_key: "0bec0cfe72089f38fb86f2def7abdf40"
server_key_iv: "1e38a1e628a9b24dad742735a59b4852"
server_key_version: "encryptionkey@0"
I20260812 03:13:36.595830 27918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "64496d67263147bf9dc2005cd49b78a0"
format_stamp: "Formatted at 2026-08-12 03:13:36 on dist-test-slave-7nm7"
server_key: "0bec0cfe72089f38fb86f2def7abdf40"
server_key_iv: "1e38a1e628a9b24dad742735a59b4852"
server_key_version: "encryptionkey@0"
I20260812 03:13:36.599794 27918 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:13:36.602427 27938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:36.603642 27918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 03:13:36.603742 27918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "64496d67263147bf9dc2005cd49b78a0"
format_stamp: "Formatted at 2026-08-12 03:13:36 on dist-test-slave-7nm7"
server_key: "0bec0cfe72089f38fb86f2def7abdf40"
server_key_iv: "1e38a1e628a9b24dad742735a59b4852"
server_key_version: "encryptionkey@0"
I20260812 03:13:36.603884 27918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:36.621161 27918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:36.624668 27918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:36.624888 27918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:36.625521 27918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:36.626415 27918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:36.626464 27918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:36.626528 27918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:36.626564 27918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:36.635622 27918 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:37373
I20260812 03:13:36.635643 28137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:37373 every 8 connection(s)
I20260812 03:13:36.636502 27918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:13:36.637830 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 27918
I20260812 03:13:36.637953 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:13:36.638222 17253 external_mini_cluster.cc:1428] Setting key 21c626d45822b512d1acd8f4dd81f56a
Aug 12 03:13:36 dist-test-slave-7nm7 krb5kdc[27492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504416, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.67@KRBTEST.COM" created.
I20260812 03:13:36.651296 27638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:36.638389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:60509 (local address 127.16.217.126:38625)
0812 03:13:36.638554 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:36.638559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:36.639573 (+ 1014us) server_negotiation.cc:407] Connection header received
0812 03:13:36.641575 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:36.641578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:36.641647 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:36.641743 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:36.643052 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.643569 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.644792 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.645038 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.645153 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:36.647602 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:36.647612 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:36.647615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:36.647637 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:36.649037 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:36.649563 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:36.649571 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:36.649572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:36.649642 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:36.650036 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:36.650039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:36.650041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:36.650314 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:36.650423 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:36.650995 (+ 572us) server_negotiation.cc:299] Negotiation successful
0812 03:13:36.651109 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:13:36.652268 28140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:36.638895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:38625 (local address 127.16.217.66:60509)
0812 03:13:36.639437 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:36.639478 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:36.641340 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:36.641938 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:36.641950 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:36.642438 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:36.642906 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:36.642920 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.643703 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.643707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:36.644649 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:36.644659 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.645192 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.645196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:36.645375 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:36.645968 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:36.645984 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:36.647439 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:36.649162 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:36.649168 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:36.649183 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:36.649450 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:36.649752 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:36.649759 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:36.649761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:36.649906 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:36.650422 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:36.650429 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:36.650731 (+ 302us) client_negotiation.cc:769] Sending connection context
0812 03:13:36.650913 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 03:13:36.651226 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":144,"threads_started":1}
I20260812 03:13:36.653604 28138 heartbeater.cc:347] Connected to a master server at 127.16.217.126:38625
I20260812 03:13:36.653870 28138 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:36.654431 28138 heartbeater.cc:511] Master 127.16.217.126:38625 requested a full tablet report, sending...
I20260812 03:13:36.655601 27571 ts_manager.cc:194] Registered new tserver with Master: 64496d67263147bf9dc2005cd49b78a0 (127.16.217.66:37373)
I20260812 03:13:36.656481 27571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:60509
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
I20260812 03:13:36.669565 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:38625
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:36.789687 28145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:36.789944 28145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:36.789988 28145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:36.794113 28145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:36.794180 28145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:36.794240 28145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:13:36.799584 28145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:38625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:36.800630 28145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:36.801602 28145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:36.811391 28152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:36.811455 28159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:36.811394 28154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:36.811517 28145 server_base.cc:1034] running on GCE node
I20260812 03:13:36.812001 28145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:36.812666 28145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:36.813859 28145 hybrid_clock.cc:648] HybridClock initialized: now 1786504416813832 us; error 38 us; skew 500 ppm
Aug 12 03:13:36 dist-test-slave-7nm7 krb5kdc[27492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504416, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:36.816721 28145 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:13:36.817515 28145 webserver.cc:460] Webserver started at http://127.16.217.67:38165/ using document root <none> and password file <none>
I20260812 03:13:36.818037 28145 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:36.818097 28145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:36.818327 28145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:36.820297 28145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "7983e922e3a248a9b67626af9deedd1a"
format_stamp: "Formatted at 2026-08-12 03:13:36 on dist-test-slave-7nm7"
server_key: "00b6fd2e7fbd0f7a7418b5d9e87a59fd"
server_key_iv: "70a01bff054657d4b7dfde39650a064f"
server_key_version: "encryptionkey@0"
I20260812 03:13:36.820809 28145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "7983e922e3a248a9b67626af9deedd1a"
format_stamp: "Formatted at 2026-08-12 03:13:36 on dist-test-slave-7nm7"
server_key: "00b6fd2e7fbd0f7a7418b5d9e87a59fd"
server_key_iv: "70a01bff054657d4b7dfde39650a064f"
server_key_version: "encryptionkey@0"
I20260812 03:13:36.824463 28145 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260812 03:13:36.826952 28172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:36.828562 28145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:36.828717 28145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "7983e922e3a248a9b67626af9deedd1a"
format_stamp: "Formatted at 2026-08-12 03:13:36 on dist-test-slave-7nm7"
server_key: "00b6fd2e7fbd0f7a7418b5d9e87a59fd"
server_key_iv: "70a01bff054657d4b7dfde39650a064f"
server_key_version: "encryptionkey@0"
I20260812 03:13:36.828838 28145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:36.853367 28145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:36.856705 28145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:36.856876 28145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:36.857548 28145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:36.858383 28145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:36.858431 28145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:36.858474 28145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:36.858489 28145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:36.868000 28145 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:36477
I20260812 03:13:36.868103 28338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:36477 every 8 connection(s)
I20260812 03:13:36.869128 28145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:13:36.877684 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 28145
I20260812 03:13:36.877856 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:13:36.878219 17253 external_mini_cluster.cc:1428] Setting key 2a9cd704559725505e329ff3c25073d7
Aug 12 03:13:36 dist-test-slave-7nm7 krb5kdc[27492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504416, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:36.883793 27638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:36.871205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:35899 (local address 127.16.217.126:38625)
0812 03:13:36.871403 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:36.871408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:36.872302 (+ 894us) server_negotiation.cc:407] Connection header received
0812 03:13:36.873626 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:36.873629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:36.873677 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:36.873754 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:36.875005 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.875537 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.876664 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.876880 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.876970 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:36.880021 (+ 3051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:36.880024 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:36.880027 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:36.880072 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:36.881595 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:36.882138 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:36.882141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:36.882142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:36.882224 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:36.882775 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:36.882779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:36.882780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:36.882969 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:36.883075 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:36.883581 (+ 506us) server_negotiation.cc:299] Negotiation successful
0812 03:13:36.883675 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 03:13:36.884661 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:36.871653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:38625 (local address 127.16.217.67:35899)
0812 03:13:36.872142 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:36.872185 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:36.873413 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:36.873923 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:36.873929 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:36.874372 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:36.874860 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:36.874874 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.875661 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.875664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:36.876539 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:36.876549 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.877018 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.877021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:36.877165 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:36.877874 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:36.877908 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:36.879835 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:36.881714 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:36.881719 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:36.881731 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:36.882005 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:36.882489 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:36.882492 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:36.882493 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:36.882654 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:36.883088 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:36.883094 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:36.883355 (+ 261us) client_negotiation.cc:769] Sending connection context
0812 03:13:36.883519 (+ 164us) client_negotiation.cc:240] Negotiation successful
0812 03:13:36.883781 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":116,"threads_started":1}
I20260812 03:13:36.885769 28343 heartbeater.cc:347] Connected to a master server at 127.16.217.126:38625
I20260812 03:13:36.885983 28343 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:36.886515 28343 heartbeater.cc:511] Master 127.16.217.126:38625 requested a full tablet report, sending...
I20260812 03:13:36.887524 27571 ts_manager.cc:194] Registered new tserver with Master: 7983e922e3a248a9b67626af9deedd1a (127.16.217.67:36477)
I20260812 03:13:36.888068 27571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:35899
I20260812 03:13:36.893275 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/test-admin.keytab.
Aug 12 03:13:36 dist-test-slave-7nm7 krb5kdc[27492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:36.910909 17253 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 03:13:36.913785 17253 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/krb5cc' of type FILE
W20260812 03:13:36.913780 28356 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:36.912314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56228 (local address 127.16.217.65:35147)
0812 03:13:36.912649 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:36.912655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:36.912680 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:13:36.912733 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:36.912739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:36.912943 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:36.913066 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:36.913598 (+ 532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56228: BlockingRecv error: recv got EOF from 127.0.0.1:56228 (error 108)
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":125,"threads_started":1}
Aug 12 03:13:36 dist-test-slave-7nm7 krb5kdc[27492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:36.914934 17253 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 03:13:36 dist-test-slave-7nm7 krb5kdc[27492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:36.915855 17253 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 03:13:36 dist-test-slave-7nm7 krb5kdc[27492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.65@KRBTEST.COM
I20260812 03:13:36.925835 28356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:36.916748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56240 (local address 127.16.217.65:35147)
0812 03:13:36.916884 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:36.916888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:36.916986 (+ 98us) server_negotiation.cc:407] Connection header received
0812 03:13:36.917142 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:36.917146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:36.917207 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:36.917294 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:36.917827 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.918683 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.919513 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.919906 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.920024 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:36.922311 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:36.922321 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:36.922339 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:36.922372 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:36.924010 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:36.924473 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:36.924479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:36.924480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:36.924557 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:36.924856 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:36.924860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:36.924861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:36.925111 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:36.925342 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:36.925535 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 03:13:36.925691 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 03:13:36.930160 28356 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:36.928925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56254 (local address 127.16.217.65:35147)
0812 03:13:36.929102 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:36.929106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:36.929223 (+ 117us) server_negotiation.cc:407] Connection header received
0812 03:13:36.929381 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:36.929386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:36.929429 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:36.929510 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:36.930046 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56254: BlockingRecv error: recv got EOF from 127.0.0.1:56254 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260812 03:13:36.930505 17253 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/krb5cc' of type FILE
Aug 12 03:13:36 dist-test-slave-7nm7 krb5kdc[27492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:36.931519 17253 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 03:13:36 dist-test-slave-7nm7 krb5kdc[27492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:36.932392 17253 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 03:13:36 dist-test-slave-7nm7 krb5kdc[27492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.65@KRBTEST.COM
I20260812 03:13:36.942140 28356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:36.933344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56256 (local address 127.16.217.65:35147)
0812 03:13:36.933456 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:36.933460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:36.933773 (+ 313us) server_negotiation.cc:407] Connection header received
0812 03:13:36.933890 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:36.933893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:36.933933 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:36.934023 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:36.934637 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.935152 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.935934 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.936367 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.936503 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:36.938996 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:36.939000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:36.939002 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:36.939035 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:36.940511 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:36.940942 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:36.940946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:36.940948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:36.941029 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:36.941309 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:36.941313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:36.941315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:36.941485 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:36.941624 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:36.941865 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 03:13:36.941976 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 03:13:36.944857 28356 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:36.943930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56264 (local address 127.16.217.65:35147)
0812 03:13:36.944100 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:36.944103 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:36.944117 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:13:36.944228 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:36.944230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:36.944265 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:36.944344 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:36.944758 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56264: BlockingRecv error: recv got EOF from 127.0.0.1:56264 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260812 03:13:36.944892 17253 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504382969880-17253-0/krb5kdc/krb5cc' of type FILE
Aug 12 03:13:36 dist-test-slave-7nm7 krb5kdc[27492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:36.945920 17253 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 03:13:36 dist-test-slave-7nm7 krb5kdc[27492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:36.946933 17253 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 03:13:36 dist-test-slave-7nm7 krb5kdc[27492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.65@KRBTEST.COM
I20260812 03:13:36.955878 28356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:36.947740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56272 (local address 127.16.217.65:35147)
0812 03:13:36.947871 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:36.947875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:36.947996 (+ 121us) server_negotiation.cc:407] Connection header received
0812 03:13:36.948126 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:36.948134 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:36.948168 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:36.948269 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:36.948745 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.949227 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.950100 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:36.950398 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:36.950533 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:36.952803 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:36.952806 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:36.952808 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:36.952829 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:36.954271 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:36.954723 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:36.954727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:36.954728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:36.954807 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:36.955105 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:36.955107 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:36.955108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:36.955260 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:36.955373 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:36.955571 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 03:13:36.955700 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:13:36.956990 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 27681
I20260812 03:13:36.971273 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 27918
I20260812 03:13:36.981441 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 28145
I20260812 03:13:37.001287 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 27518
2026-08-12T03:13:37Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2608 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:13:37 dist-test-slave-7nm7 krb5kdc[28379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:13:37 dist-test-slave-7nm7 krb5kdc[28379](info): set up 2 sockets
Aug 12 03:13:37 dist-test-slave-7nm7 krb5kdc[28379](info): commencing operation
krb5kdc: starting...
W20260812 03:13:38.311676 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:13:38 dist-test-slave-7nm7 krb5kdc[28379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:13:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:38Z Disabled control of system clock
WARNING: no policy specified for oryx/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.126.keytab.
Entry for principal oryx/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.126.keytab.
I20260812 03:13:38.462327 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:41251
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.126.keytab
--principal=oryx/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:41329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:41251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:38.588699 28406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:38.589023 28406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:38.589111 28406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:38.593138 28406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:38.593232 28406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:38.593272 28406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:38.593307 28406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:38.593339 28406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:13:38.598632 28406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:41329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:41251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.126.keytab
--principal=oryx/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:41251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:38.599785 28406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:38.600837 28406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:38.609421 28415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:38.609633 28406 server_base.cc:1034] running on GCE node
W20260812 03:13:38.609463 28413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:38.609409 28419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:38.610329 28406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:38.611385 28406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:38.612550 28406 hybrid_clock.cc:648] HybridClock initialized: now 1786504418612534 us; error 28 us; skew 500 ppm
Aug 12 03:13:38 dist-test-slave-7nm7 krb5kdc[28379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504418, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:38.615504 28406 init.cc:377] Logged in from keytab as oryx/127.16.217.126@KRBTEST.COM (short username oryx)
I20260812 03:13:38.616473 28406 webserver.cc:460] Webserver started at http://127.16.217.126:43035/ using document root <none> and password file <none>
I20260812 03:13:38.617074 28406 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:38.617173 28406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:38.617416 28406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:38.619560 28406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5"
format_stamp: "Formatted at 2026-08-12 03:13:38 on dist-test-slave-7nm7"
server_key: "6bd0a1101ca08a6263bf6405455a458a"
server_key_iv: "3f0f0acd12cee672953533ac52e4a681"
server_key_version: "encryptionkey@0"
I20260812 03:13:38.620338 28406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5"
format_stamp: "Formatted at 2026-08-12 03:13:38 on dist-test-slave-7nm7"
server_key: "6bd0a1101ca08a6263bf6405455a458a"
server_key_iv: "3f0f0acd12cee672953533ac52e4a681"
server_key_version: "encryptionkey@0"
I20260812 03:13:38.624471 28406 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:13:38.627002 28431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:38.628270 28406 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:13:38.628422 28406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5"
format_stamp: "Formatted at 2026-08-12 03:13:38 on dist-test-slave-7nm7"
server_key: "6bd0a1101ca08a6263bf6405455a458a"
server_key_iv: "3f0f0acd12cee672953533ac52e4a681"
server_key_version: "encryptionkey@0"
I20260812 03:13:38.628539 28406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:38.638476 28406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:38.641626 28406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:38.641844 28406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:38.649712 28406 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:41251
I20260812 03:13:38.650041 28526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:41251 every 8 connection(s)
I20260812 03:13:38.650836 28406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:38.653025 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 28406
I20260812 03:13:38.653169 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:13:38.653478 17253 external_mini_cluster.cc:1428] Setting key 41fa8b3a368aa04849954e2f6f706fa0
I20260812 03:13:38.654431 28527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:38.661613 28527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5: Bootstrap starting.
Aug 12 03:13:38 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM
I20260812 03:13:38.664543 28527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:38.665445 28527 log.cc:824] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:38.667601 28527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5: No bootstrap required, opened a new log
I20260812 03:13:38.669728 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:38.654971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36768 (local address 127.16.217.126:41251)
0812 03:13:38.655394 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:38.655414 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:38.655457 (+ 43us) server_negotiation.cc:407] Connection header received
0812 03:13:38.656345 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:38.656369 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:38.656788 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:38.657161 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:38.657942 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:38.659473 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:38.660679 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:38.661456 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:38.661691 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:38.664801 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:38.664808 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:38.664823 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:38.664853 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:38.666725 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:38.667217 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:38.667223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:38.667228 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:38.667339 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:38.667645 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:38.667648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:38.667650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:38.668113 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:38.668316 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:38.668627 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 03:13:38.668991 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":118,"threads_started":1}
I20260812 03:13:38.671667 28527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } }
I20260812 03:13:38.671957 28527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:38.672073 28527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63dc7a5750d74d18ba2e2a6b6d38e6c5, State: Initialized, Role: FOLLOWER
I20260812 03:13:38.672505 28527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } }
I20260812 03:13:38.672660 28527 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:38.672753 28527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:38.672873 28527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:38.673823 28527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } }
I20260812 03:13:38.674168 28527 leader_election.cc:302] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63dc7a5750d74d18ba2e2a6b6d38e6c5; no voters:
I20260812 03:13:38.674543 28527 leader_election.cc:288] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:13:38.674727 28535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:38.675062 28535 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [term 1 LEADER]: Becoming Leader. State: Replica: 63dc7a5750d74d18ba2e2a6b6d38e6c5, State: Running, Role: LEADER
I20260812 03:13:38.675375 28535 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } }
I20260812 03:13:38.675870 28527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:38.677328 28536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } }
I20260812 03:13:38.677476 28536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:38.677768 28537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63dc7a5750d74d18ba2e2a6b6d38e6c5. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } }
I20260812 03:13:38.677860 28554 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:38.677944 28537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:38.680805 28554 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:38.686726 28554 catalog_manager.cc:1347] Generated new cluster ID: 194f5c0e6bc648f3b560b57996f4cddd
I20260812 03:13:38.686805 28554 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:38.709002 28554 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:13:38.710629 28554 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:38.715849 28554 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5: Generated new TSK 0
I20260812 03:13:38.716596 28554 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.65.keytab.
Entry for principal oryx/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.65.keytab.
I20260812 03:13:38.749627 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:41251
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.65.keytab
--principal=oryx/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:41329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:38.871438 28571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:38.871716 28571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:38.871760 28571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:38.875952 28571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:38.876025 28571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:38.876085 28571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:13:38.885093 28571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:41329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.65.keytab
--principal=oryx/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:41251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:38.886860 28571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:38.888001 28571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:38.898078 28580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:38.898190 28571 server_base.cc:1034] running on GCE node
W20260812 03:13:38.897995 28583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:38.898015 28577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:38.898801 28571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:38.899456 28571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:38.900665 28571 hybrid_clock.cc:648] HybridClock initialized: now 1786504418900645 us; error 33 us; skew 500 ppm
Aug 12 03:13:38 dist-test-slave-7nm7 krb5kdc[28379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504418, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:38.904582 28571 init.cc:377] Logged in from keytab as oryx/127.16.217.65@KRBTEST.COM (short username oryx)
I20260812 03:13:38.905892 28571 webserver.cc:460] Webserver started at http://127.16.217.65:32901/ using document root <none> and password file <none>
I20260812 03:13:38.906658 28571 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:38.906761 28571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:38.907023 28571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:38.909189 28571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "f31718da75b84535bd70312147ab2aa6"
format_stamp: "Formatted at 2026-08-12 03:13:38 on dist-test-slave-7nm7"
server_key: "3dadf0690d8fdc7bf7667d0309a7f24f"
server_key_iv: "2e4b3b68009e3e3069883fa116accb7a"
server_key_version: "encryptionkey@0"
I20260812 03:13:38.909811 28571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "f31718da75b84535bd70312147ab2aa6"
format_stamp: "Formatted at 2026-08-12 03:13:38 on dist-test-slave-7nm7"
server_key: "3dadf0690d8fdc7bf7667d0309a7f24f"
server_key_iv: "2e4b3b68009e3e3069883fa116accb7a"
server_key_version: "encryptionkey@0"
I20260812 03:13:38.913879 28571 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 03:13:38.916704 28601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:38.917922 28571 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 03:13:38.918180 28571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "f31718da75b84535bd70312147ab2aa6"
format_stamp: "Formatted at 2026-08-12 03:13:38 on dist-test-slave-7nm7"
server_key: "3dadf0690d8fdc7bf7667d0309a7f24f"
server_key_iv: "2e4b3b68009e3e3069883fa116accb7a"
server_key_version: "encryptionkey@0"
I20260812 03:13:38.918391 28571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:38.943356 28571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:38.946650 28571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:38.946866 28571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:38.947536 28571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:38.948395 28571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:38.948467 28571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:38.948534 28571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:38.948577 28571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:38.957703 28571 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:34213
I20260812 03:13:38.957731 28790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:34213 every 8 connection(s)
I20260812 03:13:38.958611 28571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:13:38.967572 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 28571
I20260812 03:13:38.967738 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:13:38.968044 17253 external_mini_cluster.cc:1428] Setting key 1787da4327a5f651dd4c5729238dd865
Aug 12 03:13:38 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504418, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM
I20260812 03:13:38.973579 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:38.960612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:40797 (local address 127.16.217.126:41251)
0812 03:13:38.960763 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:38.960766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:38.961626 (+ 860us) server_negotiation.cc:407] Connection header received
0812 03:13:38.963177 (+ 1551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:38.963181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:38.963251 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:38.963368 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:38.964574 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:38.965149 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:38.966391 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:38.966632 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:38.966759 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:38.969678 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:38.969682 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:38.969684 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:38.969715 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:38.971253 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:38.971939 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:38.971944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:38.971948 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:38.972056 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:38.972425 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:38.972430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:38.972434 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:38.972683 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:38.972792 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:38.973320 (+ 528us) server_negotiation.cc:299] Negotiation successful
0812 03:13:38.973459 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:13:38.974864 28794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:38.960944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.65:40797)
0812 03:13:38.961453 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:38.961501 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:38.962923 (+ 1422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:38.963550 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:38.963556 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:38.963924 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:38.964407 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:38.964421 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:38.965291 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:38.965295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:38.966177 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:38.966188 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:38.966803 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:38.966808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:38.967001 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:38.967661 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:38.967679 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:38.969485 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:38.971383 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:38.971393 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:38.971412 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:38.971816 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:38.972172 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:38.972174 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:38.972176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:38.972320 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:38.972812 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:38.972817 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:38.973062 (+ 245us) client_negotiation.cc:769] Sending connection context
0812 03:13:38.973236 (+ 174us) client_negotiation.cc:240] Negotiation successful
0812 03:13:38.973675 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":116,"threads_started":1}
I20260812 03:13:38.976403 28791 heartbeater.cc:347] Connected to a master server at 127.16.217.126:41251
I20260812 03:13:38.976771 28791 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:38.977505 28791 heartbeater.cc:511] Master 127.16.217.126:41251 requested a full tablet report, sending...
I20260812 03:13:38.979660 28463 ts_manager.cc:194] Registered new tserver with Master: f31718da75b84535bd70312147ab2aa6 (127.16.217.65:34213)
WARNING: no policy specified for oryx/127.16.217.66@KRBTEST.COM; defaulting to no policy
I20260812 03:13:38.981325 28463 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:40797
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.66.keytab.
Entry for principal oryx/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.66.keytab.
I20260812 03:13:39.002280 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:41251
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.66.keytab
--principal=oryx/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:41329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:39.117357 28803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:39.117612 28803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:39.117655 28803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:39.122967 28803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:39.123027 28803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:39.123090 28803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:13:39.128386 28803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:41329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.66.keytab
--principal=oryx/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:41251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:39.129374 28803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:39.130424 28803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:39.140377 28814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:39.140379 28816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:39.140388 28812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:39.140848 28803 server_base.cc:1034] running on GCE node
I20260812 03:13:39.141261 28803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:39.141908 28803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:39.143136 28803 hybrid_clock.cc:648] HybridClock initialized: now 1786504419143102 us; error 43 us; skew 500 ppm
Aug 12 03:13:39 dist-test-slave-7nm7 krb5kdc[28379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504419, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:39.146003 28803 init.cc:377] Logged in from keytab as oryx/127.16.217.66@KRBTEST.COM (short username oryx)
I20260812 03:13:39.146960 28803 webserver.cc:460] Webserver started at http://127.16.217.66:35783/ using document root <none> and password file <none>
I20260812 03:13:39.147553 28803 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:39.147655 28803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:39.147890 28803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:39.150172 28803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "e91d973e36074da488958edfc2f5e863"
format_stamp: "Formatted at 2026-08-12 03:13:39 on dist-test-slave-7nm7"
server_key: "2547eda922811e5ed057f318f18e4c15"
server_key_iv: "c9243f4d7e77076b1fe9c8138814a508"
server_key_version: "encryptionkey@0"
I20260812 03:13:39.150828 28803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "e91d973e36074da488958edfc2f5e863"
format_stamp: "Formatted at 2026-08-12 03:13:39 on dist-test-slave-7nm7"
server_key: "2547eda922811e5ed057f318f18e4c15"
server_key_iv: "c9243f4d7e77076b1fe9c8138814a508"
server_key_version: "encryptionkey@0"
I20260812 03:13:39.154709 28803 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:13:39.157369 28827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:39.158865 28803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:39.159017 28803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "e91d973e36074da488958edfc2f5e863"
format_stamp: "Formatted at 2026-08-12 03:13:39 on dist-test-slave-7nm7"
server_key: "2547eda922811e5ed057f318f18e4c15"
server_key_iv: "c9243f4d7e77076b1fe9c8138814a508"
server_key_version: "encryptionkey@0"
I20260812 03:13:39.159145 28803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:39.171157 28803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:39.173928 28803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:39.174086 28803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:39.174722 28803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:39.175992 28803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:39.176059 28803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:39.176115 28803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:39.176136 28803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:39.186124 28803 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:39739
I20260812 03:13:39.186142 29018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:39739 every 8 connection(s)
I20260812 03:13:39.187155 28803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:13:39.189976 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 28803
I20260812 03:13:39.190096 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:13:39.190434 17253 external_mini_cluster.cc:1428] Setting key 0f6dc78308ab3474fa7dd932dba4663f
Aug 12 03:13:39 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504419, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.16.217.67@KRBTEST.COM; defaulting to no policy
I20260812 03:13:39.204015 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:39.189056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:48683 (local address 127.16.217.126:41251)
0812 03:13:39.189236 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:39.189241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:39.190195 (+ 954us) server_negotiation.cc:407] Connection header received
0812 03:13:39.192009 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:39.192011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:39.192186 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:39.192282 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:39.193450 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.194310 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.196345 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.196572 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.196660 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:39.199928 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:39.199932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:39.199934 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:39.199959 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:39.201689 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:39.202254 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:39.202258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:39.202260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:39.202344 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:39.202810 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:39.202814 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:39.202815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:39.203011 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:39.203121 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:39.203699 (+ 578us) server_negotiation.cc:299] Negotiation successful
0812 03:13:39.203840 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.217.67@KRBTEST.COM" created.
I20260812 03:13:39.205121 29022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:39.189412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.66:48683)
0812 03:13:39.189931 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:39.189974 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:39.191743 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:39.192433 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:39.192440 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:39.192804 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:39.193289 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:39.193302 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.194455 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.194460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:39.195965 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:39.195980 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.196755 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.196762 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:39.197046 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:39.197923 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:39.197949 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:39.199738 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:39.201814 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:39.201821 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:39.201837 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:39.202116 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:39.202442 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:39.202445 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:39.202447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:39.202686 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:39.203122 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:39.203129 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:39.203392 (+ 263us) client_negotiation.cc:769] Sending connection context
0812 03:13:39.203646 (+ 254us) client_negotiation.cc:240] Negotiation successful
0812 03:13:39.204034 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":114,"threads_started":1}
I20260812 03:13:39.206650 29019 heartbeater.cc:347] Connected to a master server at 127.16.217.126:41251
I20260812 03:13:39.206935 29019 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:39.207587 29019 heartbeater.cc:511] Master 127.16.217.126:41251 requested a full tablet report, sending...
I20260812 03:13:39.208861 28463 ts_manager.cc:194] Registered new tserver with Master: e91d973e36074da488958edfc2f5e863 (127.16.217.66:39739)
I20260812 03:13:39.209817 28463 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:48683
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.67.keytab.
Entry for principal oryx/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.67.keytab.
I20260812 03:13:39.221442 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:41251
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.67.keytab
--principal=oryx/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:41329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:39.342520 29026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:39.342789 29026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:39.342833 29026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:39.346980 29026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:39.347059 29026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:39.347123 29026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:13:39.352581 29026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:41329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.67.keytab
--principal=oryx/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:41251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:39.353658 29026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:39.354694 29026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:39.364830 29038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:39.364861 29036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:39.364830 29033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:39.365087 29026 server_base.cc:1034] running on GCE node
I20260812 03:13:39.365828 29026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:39.366588 29026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:39.367795 29026 hybrid_clock.cc:648] HybridClock initialized: now 1786504419367758 us; error 47 us; skew 500 ppm
Aug 12 03:13:39 dist-test-slave-7nm7 krb5kdc[28379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504419, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:39.370834 29026 init.cc:377] Logged in from keytab as oryx/127.16.217.67@KRBTEST.COM (short username oryx)
I20260812 03:13:39.371721 29026 webserver.cc:460] Webserver started at http://127.16.217.67:37051/ using document root <none> and password file <none>
I20260812 03:13:39.372275 29026 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:39.372335 29026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:39.372509 29026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:39.374635 29026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "4f053a481a444a4a81364a9794f487c2"
format_stamp: "Formatted at 2026-08-12 03:13:39 on dist-test-slave-7nm7"
server_key: "10ec109e98d5d6e01d6e012914bc566a"
server_key_iv: "b46d608519dbb06cc850b3cf9cdb9187"
server_key_version: "encryptionkey@0"
I20260812 03:13:39.375193 29026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "4f053a481a444a4a81364a9794f487c2"
format_stamp: "Formatted at 2026-08-12 03:13:39 on dist-test-slave-7nm7"
server_key: "10ec109e98d5d6e01d6e012914bc566a"
server_key_iv: "b46d608519dbb06cc850b3cf9cdb9187"
server_key_version: "encryptionkey@0"
I20260812 03:13:39.379462 29026 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:13:39.381955 29051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:39.383364 29026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:39.383469 29026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "4f053a481a444a4a81364a9794f487c2"
format_stamp: "Formatted at 2026-08-12 03:13:39 on dist-test-slave-7nm7"
server_key: "10ec109e98d5d6e01d6e012914bc566a"
server_key_iv: "b46d608519dbb06cc850b3cf9cdb9187"
server_key_version: "encryptionkey@0"
I20260812 03:13:39.383553 29026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:39.399219 29026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:39.402551 29026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:39.402716 29026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:39.403258 29026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:39.404052 29026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:39.404120 29026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:39.404163 29026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:39.404179 29026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:39.413580 29026 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:37525
I20260812 03:13:39.413623 29250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:37525 every 8 connection(s)
I20260812 03:13:39.414578 29026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:13:39.419405 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 29026
I20260812 03:13:39.419554 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:13:39.419883 17253 external_mini_cluster.cc:1428] Setting key 3ac63ab4b2fffcca37442b033e967c40
Aug 12 03:13:39 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504419, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM
I20260812 03:13:39.429265 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:39.416541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:46559 (local address 127.16.217.126:41251)
0812 03:13:39.416718 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:39.416723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:39.417603 (+ 880us) server_negotiation.cc:407] Connection header received
0812 03:13:39.419041 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:39.419044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:39.419094 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:39.419176 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:39.420726 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.421473 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.422857 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.423083 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.423211 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:39.425671 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:39.425674 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:39.425677 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:39.425706 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:39.427224 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:39.427753 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:39.427757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:39.427759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:39.427842 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:39.428197 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:39.428201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:39.428203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:39.428385 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:39.428496 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:39.428983 (+ 487us) server_negotiation.cc:299] Negotiation successful
0812 03:13:39.429103 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 03:13:39.430058 29256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:39.416938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.67:46559)
0812 03:13:39.417454 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:39.417497 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:39.418841 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:39.419390 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:39.419400 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:39.419920 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:39.420570 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:39.420593 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.421608 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.421612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:39.422718 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:39.422728 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.423251 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.423255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:39.423432 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:39.424042 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:39.424059 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:39.425498 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:39.427358 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:39.427363 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:39.427379 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:39.427644 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:39.427939 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:39.427943 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:39.427945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:39.428095 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:39.428496 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:39.428503 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:39.428749 (+ 246us) client_negotiation.cc:769] Sending connection context
0812 03:13:39.428942 (+ 193us) client_negotiation.cc:240] Negotiation successful
0812 03:13:39.429270 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":135,"threads_started":1}
I20260812 03:13:39.431195 29252 heartbeater.cc:347] Connected to a master server at 127.16.217.126:41251
I20260812 03:13:39.431478 29252 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:39.431998 29252 heartbeater.cc:511] Master 127.16.217.126:41251 requested a full tablet report, sending...
I20260812 03:13:39.433202 28463 ts_manager.cc:194] Registered new tserver with Master: 4f053a481a444a4a81364a9794f487c2 (127.16.217.67:37525)
I20260812 03:13:39.433943 28463 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:46559
I20260812 03:13:39.434338 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.16.217.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.217.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.217.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.125.keytab.
Entry for principal oryx/127.16.217.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.125.keytab.
I20260812 03:13:39.465729 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.125:38415
--webserver_interface=127.16.217.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.125.keytab
--principal=oryx/127.16.217.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:41329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:41251,127.16.217.125:38415
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:39.582828 29259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:39.583118 29259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:39.583161 29259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:39.587766 29259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:39.587843 29259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:39.587864 29259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:39.587881 29259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:39.587898 29259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:13:39.594208 29259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:41329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:41251,127.16.217.125:38415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.125.keytab
--principal=oryx/127.16.217.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.125:38415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.217.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
W20260812 03:13:39.594483 29259 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.16.217.126:41251,127.16.217.125:38415'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 03:13:39.595368 29259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:39.596344 29259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:39.605329 29267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:39.605341 29277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:39.605398 29259 server_base.cc:1034] running on GCE node
W20260812 03:13:39.605337 29266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:39.606073 29259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:39.607359 29259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:39.608587 29259 hybrid_clock.cc:648] HybridClock initialized: now 1786504419608555 us; error 47 us; skew 500 ppm
Aug 12 03:13:39 dist-test-slave-7nm7 krb5kdc[28379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504419, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:39.611578 29259 init.cc:377] Logged in from keytab as oryx/127.16.217.125@KRBTEST.COM (short username oryx)
I20260812 03:13:39.612548 29259 webserver.cc:460] Webserver started at http://127.16.217.125:46217/ using document root <none> and password file <none>
I20260812 03:13:39.613124 29259 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:39.613217 29259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:39.613454 29259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:39.615597 29259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/data/instance:
uuid: "82c62265ebbe4182a856f2f8759fd5ef"
format_stamp: "Formatted at 2026-08-12 03:13:39 on dist-test-slave-7nm7"
server_key: "d7a9270777f5d1f083367a1229d021b5"
server_key_iv: "099e4ce6185c982351465a197cce4f3e"
server_key_version: "encryptionkey@0"
I20260812 03:13:39.616173 29259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/wal/instance:
uuid: "82c62265ebbe4182a856f2f8759fd5ef"
format_stamp: "Formatted at 2026-08-12 03:13:39 on dist-test-slave-7nm7"
server_key: "d7a9270777f5d1f083367a1229d021b5"
server_key_iv: "099e4ce6185c982351465a197cce4f3e"
server_key_version: "encryptionkey@0"
I20260812 03:13:39.620090 29259 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:13:39.623322 29297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:39.624943 29259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:39.625118 29259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/wal
uuid: "82c62265ebbe4182a856f2f8759fd5ef"
format_stamp: "Formatted at 2026-08-12 03:13:39 on dist-test-slave-7nm7"
server_key: "d7a9270777f5d1f083367a1229d021b5"
server_key_iv: "099e4ce6185c982351465a197cce4f3e"
server_key_version: "encryptionkey@0"
I20260812 03:13:39.625258 29259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:39.645507 29259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:39.649909 29259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:39.650377 29259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:39.658658 29259 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.125:38415
I20260812 03:13:39.658946 29385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.125:38415 every 8 connection(s)
I20260812 03:13:39.659770 29259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/data/info.pb
I20260812 03:13:39.663388 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 29259
I20260812 03:13:39.663785 29388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:39.669639 29388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:13:39 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.217.125@KRBTEST.COM
Aug 12 03:13:39 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504419, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.125@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM
I20260812 03:13:39.680190 29396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:39.665223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48346 (local address 127.16.217.125:38415)
0812 03:13:39.665756 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:39.665772 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:39.665809 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:13:39.666614 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:39.666634 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:39.666959 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:39.667239 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:39.668196 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.669202 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.670372 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.670899 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.671078 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:39.674177 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:39.674189 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:39.674206 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:39.674348 (+ 142us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:39.676396 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:39.676943 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:39.676951 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:39.676958 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:39.677113 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:39.677558 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:39.677567 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:39.677569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:39.678136 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:39.678392 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:39.678679 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 03:13:39.678983 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":171,"threads_started":1}
I20260812 03:13:39.685458 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:39.671772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36782 (local address 127.16.217.126:41251)
0812 03:13:39.671983 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:39.671990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:39.672589 (+ 599us) server_negotiation.cc:407] Connection header received
0812 03:13:39.672789 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:39.672793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:39.672858 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:39.672955 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:39.674952 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.675619 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.676862 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.677275 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.677415 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:39.681033 (+ 3618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:39.681037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:39.681040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:39.681076 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:39.683535 (+ 2459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:39.684069 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:39.684072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:39.684074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:39.684167 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:39.684462 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:39.684467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:39.684468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:39.684696 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:39.684822 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:39.685162 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 03:13:39.685285 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 03:13:39.685499 29397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:39.671734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.0.0.1:36782)
0812 03:13:39.672415 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:39.672486 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:39.672675 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:39.673829 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:39.673836 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:39.674633 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:39.674806 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:39.674819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.675761 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.675765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:39.676729 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:39.676738 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.677541 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.677547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:39.677691 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:39.678576 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:39.678590 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:39.680845 (+ 2255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:39.683686 (+ 2841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:39.683695 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:39.683701 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:39.683950 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:39.684265 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:39.684268 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:39.684271 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:39.684362 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:39.684823 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:39.684830 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:39.684950 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 03:13:39.685094 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 03:13:39.685286 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"spinlock_wait_cycles":7552,"thread_start_us":139,"threads_started":1}
I20260812 03:13:39.686861 29388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:13:39 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504419, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.125@KRBTEST.COM for oryx/127.16.217.125@KRBTEST.COM
I20260812 03:13:39.695160 29397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:39.687327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38415 (local address 127.0.0.1:48356)
0812 03:13:39.687475 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:39.687491 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:39.687627 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:39.687843 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:39.687846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:39.687994 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:39.688098 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:39.688103 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.689296 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.689299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:39.689826 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:39.689834 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.690305 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.690309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:39.690398 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:39.690780 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:39.690791 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:39.692107 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:39.693778 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:39.693780 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:39.693782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:39.693973 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:39.694222 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:39.694224 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:39.694225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:39.694381 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:39.694736 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:39.694738 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:39.694817 (+ 79us) client_negotiation.cc:769] Sending connection context
0812 03:13:39.694904 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 03:13:39.695006 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 03:13:39.695178 29396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:39.687502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48356 (local address 127.16.217.125:38415)
0812 03:13:39.687625 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:39.687628 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:39.687641 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:13:39.687712 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:39.687714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:39.687746 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:39.687810 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:39.688228 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.689176 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.689931 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.690155 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.690293 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:39.692259 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:39.692265 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:39.692267 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:39.692288 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:39.693648 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:39.694061 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:39.694064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:39.694065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:39.694136 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:39.694468 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:39.694470 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:39.694471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:39.694629 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:39.694716 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:39.694975 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 03:13:39.695061 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:13:39.697904 29388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: Bootstrap starting.
I20260812 03:13:39.700220 29388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:39.700961 29388 log.cc:824] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:39.702898 29388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: No bootstrap required, opened a new log
I20260812 03:13:39.706519 29388 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } }
I20260812 03:13:39.706756 29388 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:39.706871 29388 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82c62265ebbe4182a856f2f8759fd5ef, State: Initialized, Role: FOLLOWER
I20260812 03:13:39.707315 29388 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } }
I20260812 03:13:39.707824 29399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } } }
I20260812 03:13:39.707937 29399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:13:39.708561 29388 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [sys.catalog]: configured and running, proceeding with master startup.
W20260812 03:13:39.713640 29421 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:13:39.713733 29421 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:13:39.716451 29259 master_runner.cc:428] Detected that this master 82c62265ebbe4182a856f2f8759fd5ef is joining an existing cluster
I20260812 03:13:39.716542 29259 master_runner.cc:432] Initiating AddMaster RPC to add 127.16.217.125:38415
I20260812 03:13:39.727681 29396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:39.719277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48362 (local address 127.16.217.125:38415)
0812 03:13:39.719418 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:39.719423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:39.719437 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:13:39.719472 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:39.719474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:39.719517 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:39.719614 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:39.720402 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.721156 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.722118 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.722587 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.722711 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:39.724164 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:39.724168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:39.724170 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:39.724193 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:39.725869 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:39.726367 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:39.726371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:39.726373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:39.726456 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:39.726760 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:39.726764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:39.726766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:39.727010 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:39.727128 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:39.727386 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 03:13:39.727526 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":34}
I20260812 03:13:39.727694 29433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:39.718882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38415 (local address 127.0.0.1:48362)
0812 03:13:39.719255 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:39.719269 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:39.719335 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:39.719647 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:39.719650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:39.720013 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:39.720264 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:39.720273 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.721310 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.721314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:39.721991 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:39.722001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.722733 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.722737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:39.722828 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:39.723479 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:39.723497 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:39.724037 (+ 540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:39.725975 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:39.725980 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:39.725983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:39.726232 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:39.726559 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:39.726564 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:39.726565 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:39.726650 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:39.727126 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:39.727129 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:39.727207 (+ 78us) client_negotiation.cc:769] Sending connection context
0812 03:13:39.727324 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 03:13:39.727474 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":30,"thread_start_us":103,"threads_started":1}
I20260812 03:13:39.729866 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:39.718859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36786 (local address 127.16.217.126:41251)
0812 03:13:39.719043 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:39.719049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:39.719242 (+ 193us) server_negotiation.cc:407] Connection header received
0812 03:13:39.719450 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:39.719463 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:39.719517 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:39.719614 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:39.721329 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.722080 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.723190 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.723528 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.723649 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:39.725635 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:39.725641 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:39.725644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:39.725678 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:39.727906 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:39.728423 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:39.728427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:39.728428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:39.728533 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:39.728873 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:39.728876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:39.728878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:39.729209 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:39.729305 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:39.729603 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 03:13:39.729731 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:13:39.729878 29432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:39.718803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.0.0.1:36786)
0812 03:13:39.719151 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:39.719171 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:39.719322 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:39.719653 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:39.719655 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:39.720029 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:39.720266 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:39.720273 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.722201 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.722204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:39.723057 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:39.723070 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.723662 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.723666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:39.723766 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:39.724995 (+ 1229us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:39.725010 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:39.725470 (+ 460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:39.728073 (+ 2603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:39.728077 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:39.728078 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:39.728285 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:39.728647 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:39.728651 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:39.728652 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:39.728757 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:39.729305 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:39.729308 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:39.729403 (+ 95us) client_negotiation.cc:769] Sending connection context
0812 03:13:39.729548 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 03:13:39.729692 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":73,"spinlock_wait_cycles":37120,"thread_start_us":158,"threads_started":1}
Aug 12 03:13:39 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504418, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.126@KRBTEST.COM for oryx/127.16.217.125@KRBTEST.COM
WARNING: no policy specified for oryx/127.16.217.124@KRBTEST.COM; defaulting to no policy
I20260812 03:13:39.744639 29436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:39.734958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38415 (local address 127.0.0.1:48370)
0812 03:13:39.735403 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:39.735466 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:39.735601 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:39.736077 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:39.736086 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:39.736584 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:39.736705 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:39.736711 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.737513 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.737516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:39.738429 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:39.738440 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.738917 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.738921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:39.739048 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:39.739622 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:39.739634 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:39.741177 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:39.742943 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:39.742953 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:39.742961 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:39.743199 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:39.743486 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:39.743490 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:39.743492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:39.743581 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:39.743987 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:39.743995 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:39.744129 (+ 134us) client_negotiation.cc:769] Sending connection context
0812 03:13:39.744281 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 03:13:39.744479 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":92,"threads_started":1}
I20260812 03:13:39.744638 29396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:39.735695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48370 (local address 127.16.217.125:38415)
0812 03:13:39.735799 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:39.735803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:39.735817 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:13:39.735878 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:39.735881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:39.735932 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:39.736019 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:39.736827 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.737379 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.738562 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.738798 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.738912 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:39.741318 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:39.741322 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:39.741329 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:39.741361 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:39.742810 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:39.743305 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:39.743310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:39.743312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:39.743390 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:39.743682 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:39.743685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:39.743687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:39.743864 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:39.743970 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:39.744337 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 03:13:39.744458 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:13:39.746109 28463 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.16.217.125:38415: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef"
member_type: NON_VOTER
last_known_addr {
host: "127.16.217.125"
port: 38415
}
attrs {
promote: true
}
}
dest_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5"
cas_config_opid_index: -1
I20260812 03:13:39.746827 28463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.217.124@KRBTEST.COM" created.
I20260812 03:13:39.748950 29442 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 82c62265ebbe4182a856f2f8759fd5ef (127.16.217.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } } }
I20260812 03:13:39.749406 29338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:39.750854 29338 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 1 FOLLOWER]: Refusing update from remote peer 63dc7a5750d74d18ba2e2a6b6d38e6c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 03:13:39.750993 29442 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.16.217.125:38415
I20260812 03:13:39.751534 29259 master.cc:537] Master@127.16.217.125:38415 shutting down...
I20260812 03:13:39.753922 29442 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:39.754985 29443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } } }
I20260812 03:13:39.755184 29443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: This master's current role is: LEADER
W20260812 03:13:39.755981 28434 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260812 03:13:39.756711 29259 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:39.757004 29259 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 03:13:39.756961 28434 consensus_peers.cc:487] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 -> Peer 82c62265ebbe4182a856f2f8759fd5ef (127.16.217.125:38415): Couldn't send request to peer 82c62265ebbe4182a856f2f8759fd5ef. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 03:13:39.757129 29259 tablet_replica.cc:330] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: stopping tablet replica
I20260812 03:13:39.761511 29259 master.cc:552] Master@127.16.217.125:38415 shutdown complete.
I20260812 03:13:39.761598 29259 master_runner.cc:305] Clearing existing system tablet
I20260812 03:13:39.761933 29259 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:39.761983 29259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:39.763268 29259 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:39.764173 29451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:39.764372 29259 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:39.764489 29259 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:39.764544 29259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/wal
uuid: "82c62265ebbe4182a856f2f8759fd5ef"
format_stamp: "Formatted at 2026-08-12 03:13:39 on dist-test-slave-7nm7"
server_key: "d7a9270777f5d1f083367a1229d021b5"
server_key_iv: "099e4ce6185c982351465a197cce4f3e"
server_key_version: "encryptionkey@0"
I20260812 03:13:39.766167 29259 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: Deleting tablet data with delete state TABLET_DATA_DELETED
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.217.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.124.keytab.
Entry for principal oryx/127.16.217.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.124.keytab.
I20260812 03:13:39.768946 29259 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 03:13:39.769538 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.124:35743
--webserver_interface=127.16.217.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.124.keytab
--principal=oryx/127.16.217.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:41329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:41251,127.16.217.125:38415,127.16.217.124:35743
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:13:39.771293 29259 log.cc:1196] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 03:13:39.771775 29259 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: Deleting consensus metadata
I20260812 03:13:39.771987 29259 master_runner.cc:315] Copying system tablet from 127.16.217.126:41251
I20260812 03:13:39.773506 29259 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: tablet copy: Beginning tablet copy session from remote peer at address 127.16.217.126:41251
I20260812 03:13:39.784641 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:39.774330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36800 (local address 127.16.217.126:41251)
0812 03:13:39.774519 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:39.774526 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:39.774789 (+ 263us) server_negotiation.cc:407] Connection header received
0812 03:13:39.775047 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:39.775052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:39.775117 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:39.775181 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:39.775957 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.776891 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.778442 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.778871 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.778965 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:39.780761 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:39.780769 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:39.780774 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:39.780808 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:39.782647 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:39.783213 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:39.783217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:39.783218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:39.783317 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:39.783696 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:39.783699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:39.783701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:39.783913 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:39.783983 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:39.784333 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 03:13:39.784454 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 03:13:39.785202 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:39.774119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.0.0.1:36800)
0812 03:13:39.774687 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:39.774705 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:39.774908 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:39.775361 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:39.775365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:39.775630 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:39.775799 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:39.775810 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.777070 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.777076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:39.778288 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:39.778302 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:39.779159 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:39.779165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:39.779295 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:39.780054 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:39.780076 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:39.780584 (+ 508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:39.782789 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:39.782794 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:39.782798 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:39.783078 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:39.783450 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:39.783456 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:39.783460 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:39.783576 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:39.784056 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:39.784060 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:39.784156 (+ 96us) client_negotiation.cc:769] Sending connection context
0812 03:13:39.784860 (+ 704us) client_negotiation.cc:240] Negotiation successful
0812 03:13:39.785009 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":139,"threads_started":1}
I20260812 03:13:39.786051 28500 tablet_copy_service.cc:140] P 63dc7a5750d74d18ba2e2a6b6d38e6c5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 82c62265ebbe4182a856f2f8759fd5ef ({username='oryx', principal='oryx/127.16.217.125@KRBTEST.COM'} at 127.0.0.1:36800)
I20260812 03:13:39.786302 28500 tablet_copy_service.cc:161] P 63dc7a5750d74d18ba2e2a6b6d38e6c5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 82c62265ebbe4182a856f2f8759fd5ef at {username='oryx', principal='oryx/127.16.217.125@KRBTEST.COM'} at 127.0.0.1:36800: session id = 82c62265ebbe4182a856f2f8759fd5ef-00000000000000000000000000000000
I20260812 03:13:39.788374 28500 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 03:13:39.790419 29259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 03:13:39.792928 29259 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: tablet copy: Starting download of 0 data blocks...
I20260812 03:13:39.793141 29259 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: tablet copy: Starting download of 1 WAL segments...
I20260812 03:13:39.795187 29259 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 03:13:39.797088 28500 tablet_copy_service.cc:342] P 63dc7a5750d74d18ba2e2a6b6d38e6c5: Request end of tablet copy session 82c62265ebbe4182a856f2f8759fd5ef-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.16.217.125@KRBTEST.COM'} at 127.0.0.1:36800
I20260812 03:13:39.797207 28500 tablet_copy_service.cc:433] P 63dc7a5750d74d18ba2e2a6b6d38e6c5: ending tablet copy session 82c62265ebbe4182a856f2f8759fd5ef-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 82c62265ebbe4182a856f2f8759fd5ef
I20260812 03:13:39.804582 29259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:39.806298 29462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:39.806381 29463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:39.807075 29259 server_base.cc:1034] running on GCE node
W20260812 03:13:39.807086 29465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:39.807407 29259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:39.807701 29259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:39.808853 29259 hybrid_clock.cc:648] HybridClock initialized: now 1786504419808841 us; error 29 us; skew 500 ppm
Aug 12 03:13:39 dist-test-slave-7nm7 krb5kdc[28379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504419, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:39.810683 29259 init.cc:377] Logged in from keytab as oryx/127.16.217.125@KRBTEST.COM (short username oryx)
I20260812 03:13:39.811136 29259 webserver.cc:460] Webserver started at http://127.16.217.125:45801/ using document root <none> and password file <none>
I20260812 03:13:39.811326 29259 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:39.811370 29259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:39.812682 29259 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:39.813738 29480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:39.814019 29259 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:39.814098 29259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/wal
uuid: "82c62265ebbe4182a856f2f8759fd5ef"
format_stamp: "Formatted at 2026-08-12 03:13:39 on dist-test-slave-7nm7"
server_key: "d7a9270777f5d1f083367a1229d021b5"
server_key_iv: "099e4ce6185c982351465a197cce4f3e"
server_key_version: "encryptionkey@0"
I20260812 03:13:39.814179 29259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:39.844122 29259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:39.846510 29259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:39.852363 29259 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.125:38415
I20260812 03:13:39.852401 29565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.125:38415 every 8 connection(s)
I20260812 03:13:39.852990 29259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-1/data/info.pb
I20260812 03:13:39.855070 29566 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:13:39.856170 29566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: Bootstrap starting.
I20260812 03:13:39.866917 29566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:13:39.867272 29566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: Bootstrap complete.
I20260812 03:13:39.867988 29566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } }
I20260812 03:13:39.868175 29566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 82c62265ebbe4182a856f2f8759fd5ef, State: Initialized, Role: LEARNER
I20260812 03:13:39.868319 29566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } }
I20260812 03:13:39.868745 29572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } } }
I20260812 03:13:39.868924 29566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:39.868950 29572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [sys.catalog]: This master's current role is: LEARNER
I20260812 03:13:39.872361 29600 catalog_manager.cc:1259] Loaded cluster ID: 194f5c0e6bc648f3b560b57996f4cddd
I20260812 03:13:39.872484 29600 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: loading cluster ID for follower catalog manager: success
I20260812 03:13:39.874734 29600 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: acquiring CA information for follower catalog manager: success
I20260812 03:13:39.876279 29600 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 03:13:39.908913 29453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:39.909231 29453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:39.909322 29453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:39.913453 29453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:39.913551 29453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:39.913589 29453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:39.913663 29453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:39.913702 29453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:13:39.919692 29453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:41329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:41251,127.16.217.125:38415,127.16.217.124:35743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.124.keytab
--principal=oryx/127.16.217.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.124:35743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.217.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:39.921057 29453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:39.922310 29453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:39.931797 29610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:39.931885 29613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:39.931839 29615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:39.932152 29453 server_base.cc:1034] running on GCE node
I20260812 03:13:39.932834 29453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:39.934067 29453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:39.935281 29453 hybrid_clock.cc:648] HybridClock initialized: now 1786504419935250 us; error 36 us; skew 500 ppm
Aug 12 03:13:39 dist-test-slave-7nm7 krb5kdc[28379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504419, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:39.938587 29453 init.cc:377] Logged in from keytab as oryx/127.16.217.124@KRBTEST.COM (short username oryx)
I20260812 03:13:39.939530 29453 webserver.cc:460] Webserver started at http://127.16.217.124:45087/ using document root <none> and password file <none>
I20260812 03:13:39.940124 29453 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:39.940218 29453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:39.940460 29453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:39.942684 29453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/data/instance:
uuid: "afac16e6c29649e5a414afb3b99f3c72"
format_stamp: "Formatted at 2026-08-12 03:13:39 on dist-test-slave-7nm7"
server_key: "db22d3d057899ccbecfcbe7059411fce"
server_key_iv: "3c3c8b9e30f81e65e3bd2f2cc82b09f9"
server_key_version: "encryptionkey@0"
I20260812 03:13:39.943280 29453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/wal/instance:
uuid: "afac16e6c29649e5a414afb3b99f3c72"
format_stamp: "Formatted at 2026-08-12 03:13:39 on dist-test-slave-7nm7"
server_key: "db22d3d057899ccbecfcbe7059411fce"
server_key_iv: "3c3c8b9e30f81e65e3bd2f2cc82b09f9"
server_key_version: "encryptionkey@0"
I20260812 03:13:39.948237 29453 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 03:13:39.951889 29625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:39.953469 29453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:13:39.953676 29453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/wal
uuid: "afac16e6c29649e5a414afb3b99f3c72"
format_stamp: "Formatted at 2026-08-12 03:13:39 on dist-test-slave-7nm7"
server_key: "db22d3d057899ccbecfcbe7059411fce"
server_key_iv: "3c3c8b9e30f81e65e3bd2f2cc82b09f9"
server_key_version: "encryptionkey@0"
I20260812 03:13:39.953826 29453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:39.979805 29453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:39.983632 29453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:39.983915 29453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:39.983984 28791 heartbeater.cc:503] Master 127.16.217.126:41251 was elected leader, sending a full tablet report...
I20260812 03:13:39.996954 29453 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.124:35743
I20260812 03:13:39.997443 29703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.124:35743 every 8 connection(s)
I20260812 03:13:39.998049 29453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/data/info.pb
I20260812 03:13:39.999210 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 29453
I20260812 03:13:40.001766 29704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:40.006655 29704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:13:40 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.217.124@KRBTEST.COM
Aug 12 03:13:40 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504419, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.124@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM
I20260812 03:13:40.015667 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.000911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49162 (local address 127.16.217.124:35743)
0812 03:13:40.001414 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.001433 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.001471 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:13:40.002382 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.002401 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.002818 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.003196 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.004158 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.005316 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.006483 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.006970 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.007213 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.010227 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.010263 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.010277 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.010310 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.012178 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.012671 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.012679 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.012685 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.012840 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.013178 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.013182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.013184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.013596 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.013834 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.014152 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.014603 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":152,"threads_started":1}
I20260812 03:13:40.019461 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.008718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36810 (local address 127.16.217.126:41251)
0812 03:13:40.008915 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.008921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.009218 (+ 297us) server_negotiation.cc:407] Connection header received
0812 03:13:40.009355 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.009358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.009423 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.009518 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.010477 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.011270 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.012363 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.012571 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.012706 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.015219 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.015223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.015225 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.015255 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.017106 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.018122 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.018126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.018127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.018302 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.018587 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.018589 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.018590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.018833 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.018914 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.019243 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.019346 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 03:13:40.019651 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.008551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.0.0.1:36810)
0812 03:13:40.009074 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.009123 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.009249 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.009615 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.009620 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.010034 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:40.010341 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.010350 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.011413 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.011419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.012208 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.012219 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.012719 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.012722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.012819 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.013408 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:40.013421 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:40.015046 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:40.017367 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.017385 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.017393 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.017989 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.018390 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.018393 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.018395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.018504 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.018928 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:40.018933 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:40.019032 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.019169 (+ 137us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.019326 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":255,"threads_started":1}
I20260812 03:13:40.020834 29704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:13:40 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504419, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.124@KRBTEST.COM for oryx/127.16.217.125@KRBTEST.COM
I20260812 03:13:40.030562 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.021374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38415 (local address 127.0.0.1:48374)
0812 03:13:40.021538 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.021551 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.021749 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.022184 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.022187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.022373 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:40.022469 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.022475 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.023609 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.023615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.024190 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.024200 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.024679 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.024683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.024807 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.025326 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:40.025339 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:40.026726 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:40.028925 (+ 2199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.028928 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.028929 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.029142 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.029551 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.029554 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.029555 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.029643 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.030036 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:40.030039 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:40.030141 (+ 102us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.030306 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.030435 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 03:13:40.030825 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.021517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48374 (local address 127.16.217.125:38415)
0812 03:13:40.021901 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.021904 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.021915 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:13:40.021998 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.022001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.022053 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.022169 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.022589 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.023473 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.024317 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.024559 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.024670 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.026897 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.026901 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.026903 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.026929 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.028659 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.029329 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.029334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.029335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.029424 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.029737 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.029740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.029741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.029922 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.030003 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.030484 (+ 481us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.030597 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":142,"threads_started":1}
I20260812 03:13:40.031641 29704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:13:40 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504419, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.124@KRBTEST.COM for oryx/127.16.217.124@KRBTEST.COM
I20260812 03:13:40.040311 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.032231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35743 (local address 127.0.0.1:49178)
0812 03:13:40.032415 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.032432 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.032525 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.032754 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.032757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.032895 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:40.032978 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.032984 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.034333 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.034337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.034854 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.034863 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.035368 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.035370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.035429 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.035890 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:40.035902 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:40.037167 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:40.038881 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.038884 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.038886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.039092 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.039364 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.039366 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.039367 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.039448 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.039804 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:40.039807 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:40.039899 (+ 92us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.039993 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.040139 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260812 03:13:40.040333 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.032381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49178 (local address 127.16.217.124:35743)
0812 03:13:40.032499 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.032503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.032514 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:13:40.032598 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.032600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.032633 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.032700 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.033107 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.034136 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.034981 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.035247 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.035340 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.037298 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.037301 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.037303 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.037324 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.038749 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.039184 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.039187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.039188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.039263 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.039529 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.039531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.039532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.039694 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.039770 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.040069 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.040196 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":49}
I20260812 03:13:40.043514 29704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: Bootstrap starting.
I20260812 03:13:40.046130 29704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:40.046962 29704 log.cc:824] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:40.049082 29704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: No bootstrap required, opened a new log
I20260812 03:13:40.053186 29704 raft_consensus.cc:348] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } }
I20260812 03:13:40.053428 29704 raft_consensus.cc:374] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:40.053515 29704 raft_consensus.cc:729] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afac16e6c29649e5a414afb3b99f3c72, State: Initialized, Role: FOLLOWER
I20260812 03:13:40.053989 29704 consensus_queue.cc:260] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } }
I20260812 03:13:40.054579 29711 sys_catalog.cc:455] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } } }
I20260812 03:13:40.054809 29711 sys_catalog.cc:458] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:13:40.055315 29704 sys_catalog.cc:564] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 03:13:40.061461 29725 catalog_manager.cc:1558] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:13:40.061560 29725 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:13:40.064924 29453 master_runner.cc:428] Detected that this master afac16e6c29649e5a414afb3b99f3c72 is joining an existing cluster
I20260812 03:13:40.065022 29453 master_runner.cc:432] Initiating AddMaster RPC to add 127.16.217.124:35743
I20260812 03:13:40.066366 29711 raft_consensus.cc:482] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:40.066591 29711 raft_consensus.cc:504] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } }
I20260812 03:13:40.067960 29711 leader_election.cc:288] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63dc7a5750d74d18ba2e2a6b6d38e6c5 (127.16.217.126:41251), 82c62265ebbe4182a856f2f8759fd5ef (127.16.217.125:38415)
I20260812 03:13:40.068805 28483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "afac16e6c29649e5a414afb3b99f3c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" is_pre_election: true
I20260812 03:13:40.069991 29526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "afac16e6c29649e5a414afb3b99f3c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82c62265ebbe4182a856f2f8759fd5ef" is_pre_election: true
I20260812 03:13:40.070184 29526 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 1 LEARNER]: Handling vote request from an unknown peer afac16e6c29649e5a414afb3b99f3c72
I20260812 03:13:40.070358 29526 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate afac16e6c29649e5a414afb3b99f3c72 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260812 03:13:40.070611 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 28571
I20260812 03:13:40.070796 29627 leader_election.cc:302] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: afac16e6c29649e5a414afb3b99f3c72; no voters: 63dc7a5750d74d18ba2e2a6b6d38e6c5, 82c62265ebbe4182a856f2f8759fd5ef
I20260812 03:13:40.071226 29711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:40.072804 29711 raft_consensus.cc:2726] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 03:13:40.094756 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.067392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36822 (local address 127.16.217.126:41251)
0812 03:13:40.067528 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.067536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.067937 (+ 401us) server_negotiation.cc:407] Connection header received
0812 03:13:40.068443 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.068448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.068636 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.068722 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.069469 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.070361 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.085054 (+ 14693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.085600 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.085731 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.089023 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.089031 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.089037 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.089079 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.092669 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.093241 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.093245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.093247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.093347 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.093666 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.093669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.093671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.093894 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.093969 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.094444 (+ 475us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.094575 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:13:40.096272 29736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.067401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.0.0.1:36822)
0812 03:13:40.067832 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.067845 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.068202 (+ 357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.068906 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.068909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.069192 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:40.069350 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.069360 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.070580 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.070583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.084747 (+ 14164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.084782 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.085999 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.086004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.086177 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.087579 (+ 1402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:40.087601 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:40.088809 (+ 1208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:40.092810 (+ 4001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.092814 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.092816 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.093092 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.093470 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.093473 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.093475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.093563 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.094047 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:40.094050 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:40.094157 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.095268 (+ 1111us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.095451 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":132,"threads_started":1}
I20260812 03:13:40.100394 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.068861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49190 (local address 127.16.217.124:35743)
0812 03:13:40.068974 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.068978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.087741 (+ 18763us) server_negotiation.cc:407] Connection header received
0812 03:13:40.088984 (+ 1243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.088988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.089042 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.089092 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.089818 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.090640 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.093946 (+ 3306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.094367 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.094459 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.096468 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.096475 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.096480 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.096540 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.098778 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.099203 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.099208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.099212 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.099298 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.099597 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.099600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.099601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.099787 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.099857 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.100138 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.100250 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:13:40.100937 29737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.068419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35743 (local address 127.0.0.1:49190)
0812 03:13:40.087581 (+ 19162us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.087601 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.087815 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.089243 (+ 1428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.089246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.089539 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:40.089671 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.089680 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.091099 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.091105 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.093813 (+ 2708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.093828 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.094626 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.094632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.094804 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.095491 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:40.095519 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:40.096051 (+ 532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:40.098886 (+ 2835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.098889 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.098891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.099104 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.099395 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.099397 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.099399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.099485 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.099911 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:40.099914 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:40.099987 (+ 73us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.100601 (+ 614us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.100763 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19057,"thread_start_us":127,"threads_started":1}
I20260812 03:13:40.102026 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:34213
--local_ip_for_outbound_sockets=127.16.217.65
--tserver_master_addrs=127.16.217.126:41251
--webserver_port=32901
--webserver_interface=127.16.217.65
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.65.keytab
--principal=oryx/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:41329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.217.126:41251,127.16.217.125:38415
--tserver_master_addrs=127.16.217.126:41251,127.16.217.125:38415,127.16.217.124:35743 with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:13:40 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504418, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.126@KRBTEST.COM for oryx/127.16.217.124@KRBTEST.COM
I20260812 03:13:40.130604 29436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.102742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35743 (local address 127.0.0.1:49198)
0812 03:13:40.102879 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.102896 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.102980 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.103289 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.103295 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.103509 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:40.103650 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.103660 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.104416 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.104420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.104971 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.104979 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.105444 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.105447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.105541 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.105993 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:40.106022 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:40.107313 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:40.128290 (+ 20977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.128294 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.128297 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.128629 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.129089 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.129094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.129099 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.129211 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.129813 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:40.129819 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:40.129983 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.130122 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.130364 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 03:13:40.132978 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.102923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49198 (local address 127.16.217.124:35743)
0812 03:13:40.103030 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.103034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.103046 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:13:40.103121 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.103124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.103171 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.103251 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.103806 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.104296 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.105102 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.105318 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.105428 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.107455 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.107458 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.107461 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.107484 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.128109 (+ 20625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.128838 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.128841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.128843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.128935 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.129347 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.129356 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.129360 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.129629 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.129755 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.131297 (+ 1542us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.131488 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:13:40.133014 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.088341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48376 (local address 127.16.217.125:38415)
0812 03:13:40.088536 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.088543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.095867 (+ 7324us) server_negotiation.cc:407] Connection header received
0812 03:13:40.096176 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.096179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.096248 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.096303 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.123419 (+ 27116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.124214 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.125611 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.125975 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.126113 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.128814 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.128821 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.128824 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.128854 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.130739 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.131277 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.131280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.131282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.131376 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.131781 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.131784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.131788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.132050 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.132166 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.132687 (+ 521us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.132840 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 03:13:40.133579 29739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.088209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38415 (local address 127.0.0.1:48376)
0812 03:13:40.095733 (+ 7524us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.095748 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.095989 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.096680 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.096683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.122685 (+ 26002us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:40.123150 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.123171 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.124385 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.124389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.125421 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.125437 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.126545 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.126554 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.126794 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.128130 (+ 1336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:40.128151 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:40.128629 (+ 478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:40.130879 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.130883 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.130885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.131153 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.131557 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.131560 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.131562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.131657 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.132225 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:40.132231 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:40.132354 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.132484 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.133361 (+ 877us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7442,"thread_start_us":515,"threads_started":1}
I20260812 03:13:40.134501 28463 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.16.217.124:35743: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72"
member_type: NON_VOTER
last_known_addr {
host: "127.16.217.124"
port: 35743
}
attrs {
promote: true
}
}
dest_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5"
cas_config_opid_index: 5
I20260812 03:13:40.135193 28463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } }
I20260812 03:13:40.136348 29443 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER afac16e6c29649e5a414afb3b99f3c72 (127.16.217.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } } }
I20260812 03:13:40.137617 29664 raft_consensus.cc:1261] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 1 FOLLOWER]: Refusing update from remote peer 63dc7a5750d74d18ba2e2a6b6d38e6c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 03:13:40.137980 29443 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.16.217.124:35743
I20260812 03:13:40.138209 29446 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:40.142509 29436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.136502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38415 (local address 127.0.0.1:48390)
0812 03:13:40.137180 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.137196 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.137280 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.137743 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.137746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.137766 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:40.137856 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.137865 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.139127 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.139130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.140578 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.140591 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.141896 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.141900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.141996 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.142039 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.142292 (+ 253us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.142374 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593}
I20260812 03:13:40.142942 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.136721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48390 (local address 127.16.217.125:38415)
0812 03:13:40.136902 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.136906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.137363 (+ 457us) server_negotiation.cc:407] Connection header received
0812 03:13:40.137408 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.137411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.137465 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.137546 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:40.137985 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.138994 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.140734 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.141577 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.141693 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.141830 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.142755 (+ 925us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.142832 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260812 03:13:40.143565 29664 raft_consensus.cc:858] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } }
W20260812 03:13:40.143664 29664 raft_consensus.cc:1565] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } }
I20260812 03:13:40.144011 29526 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 1 LEARNER]: Refusing update from remote peer 63dc7a5750d74d18ba2e2a6b6d38e6c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 03:13:40.144490 29442 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 03:13:40.145155 29664 raft_consensus.cc:2932] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 82c62265ebbe4182a856f2f8759fd5ef (127.16.217.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } } }
I20260812 03:13:40.146366 29453 master.cc:537] Master@127.16.217.124:35743 shutting down...
I20260812 03:13:40.146890 29526 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER afac16e6c29649e5a414afb3b99f3c72 (127.16.217.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } } }
I20260812 03:13:40.149556 29572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63dc7a5750d74d18ba2e2a6b6d38e6c5. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } } }
I20260812 03:13:40.149678 29572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [sys.catalog]: This master's current role is: LEARNER
I20260812 03:13:40.149775 29572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } } }
I20260812 03:13:40.149839 29572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [sys.catalog]: This master's current role is: LEARNER
I20260812 03:13:40.138195 29443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } } }
I20260812 03:13:40.150261 29443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:40.152097 29711 sys_catalog.cc:455] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63dc7a5750d74d18ba2e2a6b6d38e6c5. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } } }
I20260812 03:13:40.152226 29711 sys_catalog.cc:458] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 03:13:40.152416 29711 sys_catalog.cc:455] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } } }
I20260812 03:13:40.152505 29711 sys_catalog.cc:458] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 03:13:40.152192 29751 sys_catalog.cc:455] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } } }
I20260812 03:13:40.152742 29751 sys_catalog.cc:458] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 03:13:40.152881 29744 mvcc.cc:205] Tried to move back new op lower bound from 7317522099055554560 to 7317522098895237120. Current Snapshot: MvccSnapshot[applied={T|T < 7317522098929709056}]
I20260812 03:13:40.153129 29443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: NON_VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: true } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } } }
I20260812 03:13:40.153259 29443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: This master's current role is: LEADER
W20260812 03:13:40.154505 28433 consensus_peers.cc:487] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 -> Peer afac16e6c29649e5a414afb3b99f3c72 (127.16.217.124:35743): Couldn't send request to peer afac16e6c29649e5a414afb3b99f3c72. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 03:13:40.155915 29453 raft_consensus.cc:2229] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260812 03:13:40.156214 29453 raft_consensus.cc:2258] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260812 03:13:40.156337 29453 tablet_replica.cc:330] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: stopping tablet replica
I20260812 03:13:40.157274 29443 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5: attempting to promote NON_VOTER 82c62265ebbe4182a856f2f8759fd5ef to VOTER
I20260812 03:13:40.157743 29443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } }
I20260812 03:13:40.159075 29526 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 1 LEARNER]: Refusing update from remote peer 63dc7a5750d74d18ba2e2a6b6d38e6c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 03:13:40.159577 29446 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260812 03:13:40.159804 28433 consensus_peers.cc:487] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 -> Peer afac16e6c29649e5a414afb3b99f3c72 (127.16.217.124:35743): Couldn't send request to peer afac16e6c29649e5a414afb3b99f3c72. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 03:13:40.161679 29446 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 82c62265ebbe4182a856f2f8759fd5ef (127.16.217.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } } }
I20260812 03:13:40.162158 29526 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 82c62265ebbe4182a856f2f8759fd5ef (127.16.217.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } } }
I20260812 03:13:40.163244 29446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } } }
I20260812 03:13:40.163350 29446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:40.163462 29446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } } }
I20260812 03:13:40.163527 29446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:40.164390 29749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } } }
I20260812 03:13:40.164515 29749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:13:40.177951 29453 master.cc:552] Master@127.16.217.124:35743 shutdown complete.
I20260812 03:13:40.178020 29453 master_runner.cc:305] Clearing existing system tablet
I20260812 03:13:40.178553 29453 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:40.178691 29453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:40.180285 29453 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:13:40.181264 29763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:40.181545 29453 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:40.181694 29453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:40.181739 29453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/wal
uuid: "afac16e6c29649e5a414afb3b99f3c72"
format_stamp: "Formatted at 2026-08-12 03:13:39 on dist-test-slave-7nm7"
server_key: "db22d3d057899ccbecfcbe7059411fce"
server_key_iv: "3c3c8b9e30f81e65e3bd2f2cc82b09f9"
server_key_version: "encryptionkey@0"
I20260812 03:13:40.183604 29453 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:40.186028 29453 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 03:13:40.186116 29453 log.cc:1196] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 03:13:40.186545 29453 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: Deleting consensus metadata
I20260812 03:13:40.186719 29453 master_runner.cc:315] Copying system tablet from 127.16.217.126:41251
I20260812 03:13:40.187912 29453 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: tablet copy: Beginning tablet copy session from remote peer at address 127.16.217.126:41251
I20260812 03:13:40.198846 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.188625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36832 (local address 127.16.217.126:41251)
0812 03:13:40.188863 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.188868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.189443 (+ 575us) server_negotiation.cc:407] Connection header received
0812 03:13:40.189633 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.189640 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.189705 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.189759 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.190484 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.191522 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.192794 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.193148 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.193269 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.195189 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.195194 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.195197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.195225 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.196930 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.197442 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.197446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.197448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.197537 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.197880 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.197883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.197885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.198114 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.198222 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.198552 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.198686 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260812 03:13:40.198846 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.188960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.0.0.1:36832)
0812 03:13:40.189341 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.189356 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.189521 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.189910 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.189913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.190141 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:40.190343 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.190352 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.191734 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.191737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.192644 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.192656 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.193802 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.193808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.193920 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.194562 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:40.194577 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:40.195030 (+ 453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:40.197066 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.197070 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.197072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.197332 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.197653 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.197656 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.197658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.197770 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.198258 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:40.198262 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:40.198365 (+ 103us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.198478 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.198632 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":94,"threads_started":1}
I20260812 03:13:40.199915 28500 tablet_copy_service.cc:140] P 63dc7a5750d74d18ba2e2a6b6d38e6c5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer afac16e6c29649e5a414afb3b99f3c72 ({username='oryx', principal='oryx/127.16.217.124@KRBTEST.COM'} at 127.0.0.1:36832)
I20260812 03:13:40.200028 28500 tablet_copy_service.cc:161] P 63dc7a5750d74d18ba2e2a6b6d38e6c5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer afac16e6c29649e5a414afb3b99f3c72 at {username='oryx', principal='oryx/127.16.217.124@KRBTEST.COM'} at 127.0.0.1:36832: session id = afac16e6c29649e5a414afb3b99f3c72-00000000000000000000000000000000
I20260812 03:13:40.201156 28500 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 03:13:40.202564 29453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 03:13:40.205142 29453 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: tablet copy: Starting download of 0 data blocks...
I20260812 03:13:40.205345 29453 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: tablet copy: Starting download of 1 WAL segments...
I20260812 03:13:40.207204 29453 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 03:13:40.209112 28500 tablet_copy_service.cc:342] P 63dc7a5750d74d18ba2e2a6b6d38e6c5: Request end of tablet copy session afac16e6c29649e5a414afb3b99f3c72-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.16.217.124@KRBTEST.COM'} at 127.0.0.1:36832
I20260812 03:13:40.209203 28500 tablet_copy_service.cc:433] P 63dc7a5750d74d18ba2e2a6b6d38e6c5: ending tablet copy session afac16e6c29649e5a414afb3b99f3c72-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer afac16e6c29649e5a414afb3b99f3c72
I20260812 03:13:40.212638 29019 heartbeater.cc:503] Master 127.16.217.126:41251 was elected leader, sending a full tablet report...
I20260812 03:13:40.217803 29453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:40.219651 29773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:40.219732 29782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:40.220338 29453 server_base.cc:1034] running on GCE node
W20260812 03:13:40.220588 29777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:40.220824 29453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:40.221146 29453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:40.222301 29453 hybrid_clock.cc:648] HybridClock initialized: now 1786504420222285 us; error 34 us; skew 500 ppm
Aug 12 03:13:40 dist-test-slave-7nm7 krb5kdc[28379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504420, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:40.223925 29453 init.cc:377] Logged in from keytab as oryx/127.16.217.124@KRBTEST.COM (short username oryx)
I20260812 03:13:40.224462 29453 webserver.cc:460] Webserver started at http://127.16.217.124:37933/ using document root <none> and password file <none>
I20260812 03:13:40.224666 29453 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:40.224725 29453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:40.226146 29453 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:40.227303 29805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:40.227632 29453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:40.227712 29453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/wal
uuid: "afac16e6c29649e5a414afb3b99f3c72"
format_stamp: "Formatted at 2026-08-12 03:13:39 on dist-test-slave-7nm7"
server_key: "db22d3d057899ccbecfcbe7059411fce"
server_key_iv: "3c3c8b9e30f81e65e3bd2f2cc82b09f9"
server_key_version: "encryptionkey@0"
I20260812 03:13:40.227793 29453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:40.247360 29453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:40.249795 29453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:40.256484 29453 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.124:35743
I20260812 03:13:40.256517 29897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.124:35743 every 8 connection(s)
I20260812 03:13:40.257122 29453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/master-2/data/info.pb
W20260812 03:13:40.257930 29743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:40.258257 29743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:40.258324 29743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 03:13:40.258932 29899 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:13:40.259953 29899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: Bootstrap starting.
W20260812 03:13:40.264195 29743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:40.264264 29743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:40.264348 29743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:13:40.268342 29899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:13:40.268630 29899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: Bootstrap complete.
I20260812 03:13:40.269694 29899 raft_consensus.cc:348] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } }
I20260812 03:13:40.269888 29899 raft_consensus.cc:729] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: afac16e6c29649e5a414afb3b99f3c72, State: Initialized, Role: LEARNER
I20260812 03:13:40.270012 29899 consensus_queue.cc:260] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } }
I20260812 03:13:40.270455 29899 sys_catalog.cc:564] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:40.270375 29902 sys_catalog.cc:455] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } } }
I20260812 03:13:40.270678 29902 sys_catalog.cc:458] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: This master's current role is: LEARNER
I20260812 03:13:40.272487 29743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:41329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.65.keytab
--principal=oryx/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:34213
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=32901
--tserver_master_addrs=127.16.217.126:41251,127.16.217.125:38415,127.16.217.124:35743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:40.274067 29743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:40.275085 29914 catalog_manager.cc:1259] Loaded cluster ID: 194f5c0e6bc648f3b560b57996f4cddd
I20260812 03:13:40.275143 29914 catalog_manager.cc:1552] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: loading cluster ID for follower catalog manager: success
I20260812 03:13:40.275420 29743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:13:40.277382 29914 catalog_manager.cc:1574] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: acquiring CA information for follower catalog manager: success
I20260812 03:13:40.278823 29914 catalog_manager.cc:1602] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 03:13:40.286194 29916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:40.286767 29743 server_base.cc:1034] running on GCE node
W20260812 03:13:40.287015 29919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:40.287133 29917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:40.287635 29743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:40.288393 29743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:40.289580 29743 hybrid_clock.cc:648] HybridClock initialized: now 1786504420289540 us; error 46 us; skew 500 ppm
Aug 12 03:13:40 dist-test-slave-7nm7 krb5kdc[28379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504420, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:40.293452 29743 init.cc:377] Logged in from keytab as oryx/127.16.217.65@KRBTEST.COM (short username oryx)
I20260812 03:13:40.294476 29743 webserver.cc:460] Webserver started at http://127.16.217.65:32901/ using document root <none> and password file <none>
I20260812 03:13:40.295075 29743 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:40.295152 29743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:40.299516 29743 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:40.301761 29933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:40.303220 29743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:40.303360 29743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "f31718da75b84535bd70312147ab2aa6"
format_stamp: "Formatted at 2026-08-12 03:13:38 on dist-test-slave-7nm7"
server_key: "3dadf0690d8fdc7bf7667d0309a7f24f"
server_key_iv: "2e4b3b68009e3e3069883fa116accb7a"
server_key_version: "encryptionkey@0"
I20260812 03:13:40.303926 29743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:40.317049 29743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:40.321018 29743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:40.321310 29743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:40.322074 29743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:40.323052 29743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:40.323127 29743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:40.323201 29743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:40.323252 29743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:40.337766 29743 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:34213
I20260812 03:13:40.337878 30131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:34213 every 8 connection(s)
I20260812 03:13:40.339021 29743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:13:40.344338 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 29743
I20260812 03:13:40.344551 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 28803
Aug 12 03:13:40 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504420, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.124@KRBTEST.COM
Aug 12 03:13:40 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504420, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.125@KRBTEST.COM
I20260812 03:13:40.358575 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.340985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:46221 (local address 127.16.217.125:38415)
0812 03:13:40.341159 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.341166 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.342957 (+ 1791us) server_negotiation.cc:407] Connection header received
0812 03:13:40.344050 (+ 1093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.344053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.344107 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.344201 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.348372 (+ 4171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.349141 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.350304 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.350633 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.350718 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.353617 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.353622 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.353624 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.353655 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.355731 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.356473 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.356476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.356479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.356567 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.356978 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.357007 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.357009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.357235 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.357323 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.358133 (+ 810us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.358388 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:13:40.359663 30142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.341748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35743 (local address 127.16.217.65:38001)
0812 03:13:40.342477 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.342532 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.343544 (+ 1012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.344650 (+ 1106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.344656 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.344887 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:40.345538 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.345568 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.346727 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.346730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.347918 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.347930 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.349522 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.349526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.349717 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.350440 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:40.350461 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:40.353404 (+ 2943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:40.356253 (+ 2849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.356256 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.356258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.356505 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.356816 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.356819 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.356821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.356899 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.357383 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:40.357390 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:40.357829 (+ 439us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.358055 (+ 226us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.358547 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":41,"spinlock_wait_cycles":28928,"thread_start_us":145,"threads_started":1}
I20260812 03:13:40.359726 30143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.341748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38415 (local address 127.16.217.65:46221)
0812 03:13:40.342869 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.342884 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.343544 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.344236 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.344240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.344660 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:40.348231 (+ 3571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.348239 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.349278 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.349281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.350132 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.350146 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.350874 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.350878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.350976 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.351646 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:40.351663 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:40.353404 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:40.355890 (+ 2486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.355899 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.355919 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.356338 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.356677 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.356681 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.356683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.356864 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.357383 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:40.357390 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:40.357829 (+ 439us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.359028 (+ 1199us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.359216 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1035,"mutex_wait_us":128,"spinlock_wait_cycles":9856,"thread_start_us":226,"threads_started":1}
I20260812 03:13:40.360605 30145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.342440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:38001 (local address 127.16.217.124:35743)
0812 03:13:40.342870 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.342876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.342896 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:13:40.344342 (+ 1446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.344345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.344406 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.344498 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.345739 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.346577 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.348059 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.348434 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.348550 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.353626 (+ 5076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.353629 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.353632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.353657 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.356111 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.356620 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.356623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.356625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.356713 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.357002 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.357006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.357007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.357235 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.357323 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.360274 (+ 2951us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.360435 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":101,"threads_started":1}
I20260812 03:13:40.365674 30137 heartbeater.cc:347] Connected to a master server at 127.16.217.125:38415
I20260812 03:13:40.366065 30137 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:40.366307 30134 heartbeater.cc:347] Connected to a master server at 127.16.217.124:35743
I20260812 03:13:40.366442 30134 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:40.366946 30137 heartbeater.cc:511] Master 127.16.217.125:38415 requested a full tablet report, sending...
I20260812 03:13:40.367224 30134 heartbeater.cc:511] Master 127.16.217.124:35743 requested a full tablet report, sending...
Aug 12 03:13:40 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504420, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM
I20260812 03:13:40.368831 29838 ts_manager.cc:194] Registered new tserver with Master: f31718da75b84535bd70312147ab2aa6 (127.16.217.65:34213)
I20260812 03:13:40.369823 29509 ts_manager.cc:194] Registered new tserver with Master: f31718da75b84535bd70312147ab2aa6 (127.16.217.65:34213)
I20260812 03:13:40.372727 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.341297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:51367 (local address 127.16.217.126:41251)
0812 03:13:40.341446 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.341450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.343009 (+ 1559us) server_negotiation.cc:407] Connection header received
0812 03:13:40.343767 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.343770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.343830 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.343912 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.346069 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.347206 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.365670 (+ 18464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.366016 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.366157 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.369143 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.369148 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.369150 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.369183 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.370883 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.371447 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.371453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.371458 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.371561 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.371846 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.371849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.371851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.372052 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.372164 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.372458 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.372577 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 03:13:40.372745 30144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.341825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.65:51367)
0812 03:13:40.342913 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.342926 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.343547 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.344091 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.344098 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.344666 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:40.345899 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.345914 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.347393 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.347399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.349342 (+ 1943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.349352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.366224 (+ 16872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.366260 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.366601 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.367222 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:40.367240 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:40.368960 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:40.371015 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.371019 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.371021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.371325 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.371669 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.371671 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.371673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.371741 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.372154 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:40.372157 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:40.372260 (+ 103us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.372381 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.372555 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1020,"mutex_wait_us":1753,"thread_start_us":239,"threads_started":1}
I20260812 03:13:40.373932 30136 heartbeater.cc:347] Connected to a master server at 127.16.217.126:41251
I20260812 03:13:40.374051 30136 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:40.374356 30136 heartbeater.cc:511] Master 127.16.217.126:41251 requested a full tablet report, sending...
I20260812 03:13:40.375236 28463 ts_manager.cc:194] Re-registered known tserver with Master: f31718da75b84535bd70312147ab2aa6 (127.16.217.65:34213)
I20260812 03:13:40.375910 28463 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:51367
I20260812 03:13:40.377372 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:39739
--local_ip_for_outbound_sockets=127.16.217.66
--tserver_master_addrs=127.16.217.126:41251
--webserver_port=35783
--webserver_interface=127.16.217.66
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.66.keytab
--principal=oryx/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:41329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.217.126:41251,127.16.217.125:38415
--tserver_master_addrs=127.16.217.126:41251,127.16.217.125:38415,127.16.217.124:35743 with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:13:40.436513 29252 heartbeater.cc:503] Master 127.16.217.126:41251 was elected leader, sending a full tablet report...
W20260812 03:13:40.499454 30146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:40.499691 30146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:40.499732 30146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:40.503635 30146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:40.503690 30146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:40.503782 30146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:13:40.509021 30146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:41329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.66.keytab
--principal=oryx/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:39739
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=35783
--tserver_master_addrs=127.16.217.126:41251,127.16.217.125:38415,127.16.217.124:35743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:40.509985 30146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:40.510988 30146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:40.520779 30161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:40.520846 30153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:40.520785 30157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:40.520920 30146 server_base.cc:1034] running on GCE node
I20260812 03:13:40.521461 30146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:40.522087 30146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:40.523296 30146 hybrid_clock.cc:648] HybridClock initialized: now 1786504420523280 us; error 28 us; skew 500 ppm
Aug 12 03:13:40 dist-test-slave-7nm7 krb5kdc[28379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504420, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:40.526362 30146 init.cc:377] Logged in from keytab as oryx/127.16.217.66@KRBTEST.COM (short username oryx)
I20260812 03:13:40.527293 30146 webserver.cc:460] Webserver started at http://127.16.217.66:35783/ using document root <none> and password file <none>
I20260812 03:13:40.527921 30146 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:40.528020 30146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:40.532286 30146 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:13:40.534711 30173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:40.535944 30146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:40.536096 30146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "e91d973e36074da488958edfc2f5e863"
format_stamp: "Formatted at 2026-08-12 03:13:39 on dist-test-slave-7nm7"
server_key: "2547eda922811e5ed057f318f18e4c15"
server_key_iv: "c9243f4d7e77076b1fe9c8138814a508"
server_key_version: "encryptionkey@0"
I20260812 03:13:40.536638 30146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:40.576138 30146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:40.579777 30146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:40.579974 30146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:40.580554 30146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:40.581403 30146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:40.581451 30146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:40.581494 30146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:40.581519 30146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:40.591341 30146 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:39739
I20260812 03:13:40.591372 30373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:39739 every 8 connection(s)
I20260812 03:13:40.592381 30146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:13:40.595718 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 30146
I20260812 03:13:40.595893 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 29026
Aug 12 03:13:40 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504420, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.124@KRBTEST.COM
Aug 12 03:13:40 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504420, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.125@KRBTEST.COM
I20260812 03:13:40.611505 30145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.594877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:38837 (local address 127.16.217.124:35743)
0812 03:13:40.594998 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.595026 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.596156 (+ 1130us) server_negotiation.cc:407] Connection header received
0812 03:13:40.597650 (+ 1494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.597657 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.597732 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.597800 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.599803 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.600588 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.602363 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.602728 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.602803 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.606439 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.606446 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.606451 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.606485 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.608999 (+ 2514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.609654 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.609660 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.609664 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.609743 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.610112 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.610118 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.610122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.610379 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.610448 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.611122 (+ 674us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.611270 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:13:40.611847 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.594376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:59359 (local address 127.16.217.125:38415)
0812 03:13:40.594514 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.594519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.600034 (+ 5515us) server_negotiation.cc:407] Connection header received
0812 03:13:40.600210 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.600213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.600263 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.600312 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.601262 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.602029 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.603158 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.603468 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.603562 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.607193 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.607199 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.607204 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.607236 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.609239 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.609926 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.609933 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.609937 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.610050 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.610482 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.610489 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.610493 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.610829 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.610938 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.611400 (+ 462us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.611591 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:13:40.613247 30383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.594919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35743 (local address 127.16.217.66:38837)
0812 03:13:40.595998 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.596059 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.597332 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.598360 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.598374 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.599053 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:40.599639 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.599653 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.600740 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.600746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.602210 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.602217 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.603018 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.603022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.603262 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.603979 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:40.604005 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:40.606179 (+ 2174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:40.609152 (+ 2973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.609160 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.609181 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.609536 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.609845 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.609850 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.609854 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.610020 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.610526 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:40.610535 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:40.610866 (+ 331us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.611747 (+ 881us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.612123 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"mutex_wait_us":81,"thread_start_us":137,"threads_started":1}
I20260812 03:13:40.613260 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.594919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38415 (local address 127.16.217.66:59359)
0812 03:13:40.599943 (+ 5024us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.599957 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.600104 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.600439 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.600441 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.600690 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:40.600862 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.600869 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.602180 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.602183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.603029 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.603036 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.603745 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.603752 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.603879 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.604414 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:40.604429 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:40.606955 (+ 2526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:40.609410 (+ 2455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.609417 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.609422 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.609777 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.610187 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.610193 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.610198 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.610345 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.611014 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:40.611017 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:40.611150 (+ 133us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.612113 (+ 963us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.612315 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4933,"mutex_wait_us":164,"spinlock_wait_cycles":20864,"thread_start_us":123,"threads_started":1}
I20260812 03:13:40.617518 30374 heartbeater.cc:347] Connected to a master server at 127.16.217.124:35743
I20260812 03:13:40.617812 30374 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:40.618522 30374 heartbeater.cc:511] Master 127.16.217.124:35743 requested a full tablet report, sending...
I20260812 03:13:40.619580 30378 heartbeater.cc:347] Connected to a master server at 127.16.217.125:38415
I20260812 03:13:40.619694 30378 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:40.619915 30378 heartbeater.cc:511] Master 127.16.217.125:38415 requested a full tablet report, sending...
I20260812 03:13:40.620772 29509 ts_manager.cc:194] Registered new tserver with Master: e91d973e36074da488958edfc2f5e863 (127.16.217.66:39739)
I20260812 03:13:40.622613 29838 ts_manager.cc:194] Registered new tserver with Master: e91d973e36074da488958edfc2f5e863 (127.16.217.66:39739)
Aug 12 03:13:40 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504420, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM
I20260812 03:13:40.625422 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:37525
--local_ip_for_outbound_sockets=127.16.217.67
--tserver_master_addrs=127.16.217.126:41251
--webserver_port=37051
--webserver_interface=127.16.217.67
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.67.keytab
--principal=oryx/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:41329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.217.126:41251,127.16.217.125:38415
--tserver_master_addrs=127.16.217.126:41251,127.16.217.125:38415,127.16.217.124:35743 with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:13:40.627183 30381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.594919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.66:40091)
0812 03:13:40.617755 (+ 22836us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.617774 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.617904 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.618303 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.618306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.618524 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:40.618642 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.618650 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.619858 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.619862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.620631 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.620641 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.621179 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.621186 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.621325 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.621855 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:40.621877 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:40.623496 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:40.625400 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.625402 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.625404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.625714 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.626075 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.626077 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.626078 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.626156 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.626643 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:40.626645 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:40.626722 (+ 77us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.626799 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.626908 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":163,"threads_started":1}
I20260812 03:13:40.627302 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.594401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:40091 (local address 127.16.217.126:41251)
0812 03:13:40.594560 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.594567 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.617875 (+ 23308us) server_negotiation.cc:407] Connection header received
0812 03:13:40.618069 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.618075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.618147 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.618262 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.618754 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.619728 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.620765 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.621042 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.621145 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.623649 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.623653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.623655 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.623678 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.625272 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.625838 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.625845 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.625849 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.625947 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.626277 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.626283 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.626287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.626537 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.626657 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.626903 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.627074 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:13:40.627856 30376 heartbeater.cc:347] Connected to a master server at 127.16.217.126:41251
I20260812 03:13:40.627959 30376 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:40.628158 30376 heartbeater.cc:511] Master 127.16.217.126:41251 requested a full tablet report, sending...
I20260812 03:13:40.628835 28463 ts_manager.cc:194] Re-registered known tserver with Master: e91d973e36074da488958edfc2f5e863 (127.16.217.66:39739)
I20260812 03:13:40.629455 28463 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:40091
W20260812 03:13:40.742674 30384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:40.742931 30384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:40.743041 30384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 03:13:40.747977 30391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.743930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35743 (local address 127.0.0.1:49202)
0812 03:13:40.744229 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.744242 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.744359 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.744654 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.744656 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.744672 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:40.744775 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.744779 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.745790 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.745793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.746679 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.746689 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.747602 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.747605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.747688 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.747713 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.747776 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.747839 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":85,"threads_started":1}
I20260812 03:13:40.747982 30145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.744017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49202 (local address 127.16.217.124:35743)
0812 03:13:40.744175 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.744180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.744327 (+ 147us) server_negotiation.cc:407] Connection header received
0812 03:13:40.744441 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.744444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.744516 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.744631 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:40.744875 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.745672 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.746800 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.747455 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.747584 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.747718 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.747792 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.747857 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260812 03:13:40.748559 30384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:40.748611 30384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:40.748715 30384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:13:40.748718 29852 raft_consensus.cc:1203] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 03:13:40.749032 29902 sys_catalog.cc:455] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63dc7a5750d74d18ba2e2a6b6d38e6c5. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } } }
I20260812 03:13:40.749217 29902 sys_catalog.cc:458] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: This master's current role is: LEARNER
I20260812 03:13:40.754380 30384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:41329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.67.keytab
--principal=oryx/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:37525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=37051
--tserver_master_addrs=127.16.217.126:41251,127.16.217.125:38415,127.16.217.124:35743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:40.755424 30384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:40.756371 30384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:40.766608 30395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:40.766571 30394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:40.766764 30399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:40.767225 30384 server_base.cc:1034] running on GCE node
I20260812 03:13:40.767592 30384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:40.768189 30384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:40.769358 30384 hybrid_clock.cc:648] HybridClock initialized: now 1786504420769325 us; error 44 us; skew 500 ppm
Aug 12 03:13:40 dist-test-slave-7nm7 krb5kdc[28379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504420, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:40.772710 30384 init.cc:377] Logged in from keytab as oryx/127.16.217.67@KRBTEST.COM (short username oryx)
I20260812 03:13:40.773768 30384 webserver.cc:460] Webserver started at http://127.16.217.67:37051/ using document root <none> and password file <none>
I20260812 03:13:40.774467 30384 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:40.774581 30384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:40.779132 30384 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:40.781580 30410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:40.783178 30384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:13:40.783357 30384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "4f053a481a444a4a81364a9794f487c2"
format_stamp: "Formatted at 2026-08-12 03:13:39 on dist-test-slave-7nm7"
server_key: "10ec109e98d5d6e01d6e012914bc566a"
server_key_iv: "b46d608519dbb06cc850b3cf9cdb9187"
server_key_version: "encryptionkey@0"
I20260812 03:13:40.783993 30384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:40.800652 30384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:40.804932 30384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:40.805171 30384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:40.805810 30384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:40.806833 30384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:40.806910 30384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:40.806984 30384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:40.807035 30384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:40.819231 30384 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:37525
I20260812 03:13:40.819253 30600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:37525 every 8 connection(s)
I20260812 03:13:40.820399 30384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:13:40.823868 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 30384
Aug 12 03:13:40 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504420, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM
Aug 12 03:13:40 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504420, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.125@KRBTEST.COM
I20260812 03:13:40.839324 30627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.826333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48406 (local address 127.16.217.125:38415)
0812 03:13:40.826614 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.826621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.826635 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:13:40.826674 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.826680 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.826727 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.826819 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.827874 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.828643 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.829839 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.830275 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.830372 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.832223 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.832227 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.832229 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.832257 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.835238 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.837561 (+ 2323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.837565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.837566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.837660 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.838004 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.838009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.838014 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.838344 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.838490 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.838952 (+ 462us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.839103 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":96,"threads_started":1}
Aug 12 03:13:40 dist-test-slave-7nm7 krb5kdc[28379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504420, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.124@KRBTEST.COM
I20260812 03:13:40.841113 30629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.827231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49212 (local address 127.16.217.124:35743)
0812 03:13:40.829251 (+ 2020us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.829255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.829267 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:13:40.829303 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.829306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.829348 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.829442 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.830338 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.831076 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.832631 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.834181 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.834366 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.836618 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.836621 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.836625 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.836658 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.838672 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.839151 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.839154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.839157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.839237 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.839721 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.839724 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.839727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.839943 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.840059 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.840676 (+ 617us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.840834 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1941,"thread_start_us":72,"threads_started":1}
I20260812 03:13:40.843750 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.822352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:35739 (local address 127.16.217.125:38415)
0812 03:13:40.822498 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.822503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.824730 (+ 2227us) server_negotiation.cc:407] Connection header received
0812 03:13:40.825608 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.825611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.825673 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.825754 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.827807 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.828635 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.832130 (+ 3495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.832987 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.833430 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.837950 (+ 4520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.837955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.837959 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.837991 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.839761 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.841652 (+ 1891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.841658 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.841660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.841754 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.842186 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.842189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.842191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.842465 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.842558 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.843465 (+ 907us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.843595 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":56}
I20260812 03:13:40.844307 30145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.822449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:55769 (local address 127.16.217.124:35743)
0812 03:13:40.823960 (+ 1511us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.823966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.824218 (+ 252us) server_negotiation.cc:407] Connection header received
0812 03:13:40.825487 (+ 1269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.825491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.825567 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.825626 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.833919 (+ 8293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.834796 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.836240 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.836625 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.836725 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.840340 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.840344 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.840347 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.840371 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.842063 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.842876 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.842880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.842882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.842965 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.843263 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.843266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.843268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.843468 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.843566 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.844044 (+ 478us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.844161 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1420}
I20260812 03:13:40.844911 30610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.822877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.67:60443)
0812 03:13:40.824031 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.824099 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.825234 (+ 1135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.825971 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.825980 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.826715 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:40.827601 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.827623 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.828772 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.828775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.830305 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.830319 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.831696 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.831700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.831962 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.832553 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:40.832575 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:40.835608 (+ 3033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:40.841225 (+ 5617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.841229 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.841231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.842044 (+ 813us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.842384 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.842386 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.842388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.842517 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.843056 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:40.843061 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:40.843183 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.843281 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.843629 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1044,"mutex_wait_us":173,"spinlock_wait_cycles":11648,"thread_start_us":208,"threads_started":1}
I20260812 03:13:40.844959 30616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.823602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35743 (local address 127.16.217.67:55769)
0812 03:13:40.824113 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.824125 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.825232 (+ 1107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.825971 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.825980 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.832357 (+ 6377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:40.833734 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.833743 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.835015 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.835024 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.836045 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.836059 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.836911 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.836916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.837123 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.838420 (+ 1297us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:40.838439 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:40.840189 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:40.842529 (+ 2340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.842534 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.842536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.842760 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.843077 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.843079 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.843081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.843160 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.843626 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:40.843629 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:40.843849 (+ 220us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.844513 (+ 664us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.844644 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"spinlock_wait_cycles":17024,"thread_start_us":94,"threads_started":1}
I20260812 03:13:40.845474 30614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.822877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38415 (local address 127.16.217.67:35739)
0812 03:13:40.824614 (+ 1737us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.824630 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.825245 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.825970 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.825981 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.826733 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:40.827601 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.827625 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.828792 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.828795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.831992 (+ 3197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.832002 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.833586 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.833590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.833756 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.834430 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:40.834445 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:40.837727 (+ 3282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:40.840759 (+ 3032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.840767 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.840790 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.841525 (+ 735us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.841867 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:40.841871 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:40.841873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:40.842076 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:40.842615 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:40.842622 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:40.842981 (+ 359us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.843216 (+ 235us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.844212 (+ 996us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":2982,"spinlock_wait_cycles":28928,"thread_start_us":164,"threads_started":1}
I20260812 03:13:40.845710 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.823137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:60443 (local address 127.16.217.126:41251)
0812 03:13:40.823309 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.823313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.824267 (+ 954us) server_negotiation.cc:407] Connection header received
0812 03:13:40.825571 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.825574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.825626 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.825711 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.827772 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.828575 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.830429 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.830767 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.831201 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.835832 (+ 4631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.835837 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.835841 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.835872 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.841100 (+ 5228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.842171 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.842174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.842176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.842291 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.842649 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.842656 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.842660 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.842901 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.843001 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.845384 (+ 2383us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.845519 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":78}
I20260812 03:13:40.846339 30602 heartbeater.cc:347] Connected to a master server at 127.16.217.124:35743
I20260812 03:13:40.846431 30605 heartbeater.cc:347] Connected to a master server at 127.16.217.125:38415
I20260812 03:13:40.846515 30632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.827621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36842 (local address 127.16.217.126:41251)
0812 03:13:40.828165 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.828169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.828862 (+ 693us) server_negotiation.cc:407] Connection header received
0812 03:13:40.828901 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.828906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.828956 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.829048 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:40.830409 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.831141 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.833486 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.833846 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.833961 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.837592 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:40.837596 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:40.837599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:40.837628 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:40.843288 (+ 5660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.843782 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.843786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.843787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.844025 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:40.844329 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:40.844333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:40.844335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:40.844669 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:40.844817 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.846151 (+ 1334us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.846313 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":76,"threads_started":1}
I20260812 03:13:40.846630 30602 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:40.846815 30604 heartbeater.cc:347] Connected to a master server at 127.16.217.126:41251
I20260812 03:13:40.846907 30604 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:40.846629 30605 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:40.847270 30605 heartbeater.cc:511] Master 127.16.217.125:38415 requested a full tablet report, sending...
I20260812 03:13:40.847268 30604 heartbeater.cc:511] Master 127.16.217.126:41251 requested a full tablet report, sending...
I20260812 03:13:40.847771 30602 heartbeater.cc:511] Master 127.16.217.124:35743 requested a full tablet report, sending...
I20260812 03:13:40.848476 29838 ts_manager.cc:194] Registered new tserver with Master: 4f053a481a444a4a81364a9794f487c2 (127.16.217.67:37525)
I20260812 03:13:40.848476 29509 ts_manager.cc:194] Registered new tserver with Master: 4f053a481a444a4a81364a9794f487c2 (127.16.217.67:37525)
I20260812 03:13:40.848961 28463 ts_manager.cc:194] Re-registered known tserver with Master: 4f053a481a444a4a81364a9794f487c2 (127.16.217.67:37525)
I20260812 03:13:40.849658 28463 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:60443
I20260812 03:13:40.849763 28461 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:13:40.853428 28461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:13:40.856063 29443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: NON_VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: true } } }
I20260812 03:13:40.856199 29443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:40.858271 30637 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5: attempting to promote NON_VOTER afac16e6c29649e5a414afb3b99f3c72 to VOTER
I20260812 03:13:40.858659 30637 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: false } }
I20260812 03:13:40.859803 29852 raft_consensus.cc:1261] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 1 LEARNER]: Refusing update from remote peer 63dc7a5750d74d18ba2e2a6b6d38e6c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 03:13:40.860016 29526 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 1 FOLLOWER]: Refusing update from remote peer 63dc7a5750d74d18ba2e2a6b6d38e6c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 03:13:40.860162 29443 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 03:13:40.860550 29443 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 03:13:40.862015 30637 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, afac16e6c29649e5a414afb3b99f3c72 (127.16.217.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: false } } }
I20260812 03:13:40.862840 29852 raft_consensus.cc:2932] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, afac16e6c29649e5a414afb3b99f3c72 (127.16.217.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: false } } }
I20260812 03:13:40.862962 29526 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, afac16e6c29649e5a414afb3b99f3c72 (127.16.217.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: false } } }
I20260812 03:13:40.863905 30636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: false } } }
I20260812 03:13:40.864108 30636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:40.864262 30636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: false } } }
I20260812 03:13:40.864346 30636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:40.863926 30637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: false } } }
I20260812 03:13:40.864571 30637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:40.864504 29902 sys_catalog.cc:455] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: false } } }
I20260812 03:13:40.864619 29902 sys_catalog.cc:458] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:13:40.865417 30655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: false } } }
I20260812 03:13:40.865612 30655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:13:40.879092 30391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.873601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:34213 (local address 127.0.0.1:51606)
0812 03:13:40.873951 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.873970 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.874058 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.874795 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.874798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.874818 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:40.874924 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.874934 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.876443 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.876446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.877669 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.877680 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.878695 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.878698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.878783 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.878805 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.878872 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.878948 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268}
I20260812 03:13:40.879127 30657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.873916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51606 (local address 127.16.217.65:34213)
0812 03:13:40.874224 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.874271 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.874300 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:13:40.874393 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.874401 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.874644 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.874833 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:40.875095 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.876323 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.877779 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.878559 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.878696 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.878860 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.878915 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.879004 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":77,"threads_started":1}
I20260812 03:13:40.879535 30659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.874172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:37525 (local address 127.0.0.1:50540)
0812 03:13:40.874525 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.874541 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.874637 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.875380 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.875383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.875405 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:40.875512 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.875519 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.876853 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.876856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.877944 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.877951 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.879006 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.879010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.879098 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.879122 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.879172 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.879261 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":137,"threads_started":1}
I20260812 03:13:40.880250 30661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.874608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50540 (local address 127.16.217.67:37525)
0812 03:13:40.874904 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.874918 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.874942 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:13:40.875019 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.875025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.875230 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.875644 (+ 414us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:40.875740 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.876721 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.878063 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.878860 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.879790 (+ 930us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.879947 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.880000 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.880099 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":116,"threads_started":1}
I20260812 03:13:40.882814 30662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.875066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:39739 (local address 127.0.0.1:38178)
0812 03:13:40.875314 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.875329 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.875424 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.877197 (+ 1773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.877200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.877220 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:40.877309 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.877319 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.880210 (+ 2891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.880214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.881345 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.881356 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.882408 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.882412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.882509 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.882535 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.882621 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.882692 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":104,"threads_started":1}
I20260812 03:13:40.882903 30663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.875213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38178 (local address 127.16.217.66:39739)
0812 03:13:40.876641 (+ 1428us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.876705 (+ 64us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.876726 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:13:40.876786 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.876789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.877018 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.877218 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:40.877406 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.879135 (+ 1729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.881494 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.882297 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.882401 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.882530 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.882670 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.882766 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1318,"thread_start_us":128,"threads_started":1}
I20260812 03:13:40.882943 30498 tablet_service.cc:1467] Processing CreateTablet for tablet 545396670dc34bf293d0c6cdd3416807 (DEFAULT_TABLE table=test-table [id=13255b1c49a0401e87ec54ba43e5db35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:40.882941 30020 tablet_service.cc:1467] Processing CreateTablet for tablet 545396670dc34bf293d0c6cdd3416807 (DEFAULT_TABLE table=test-table [id=13255b1c49a0401e87ec54ba43e5db35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:40.883939 30020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 545396670dc34bf293d0c6cdd3416807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:40.883940 30498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 545396670dc34bf293d0c6cdd3416807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:40.890429 30254 tablet_service.cc:1467] Processing CreateTablet for tablet 545396670dc34bf293d0c6cdd3416807 (DEFAULT_TABLE table=test-table [id=13255b1c49a0401e87ec54ba43e5db35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:40.891868 30254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 545396670dc34bf293d0c6cdd3416807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:40.892519 30666 tablet_bootstrap.cc:492] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6: Bootstrap starting.
I20260812 03:13:40.893766 30667 tablet_bootstrap.cc:492] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2: Bootstrap starting.
I20260812 03:13:40.894799 30666 tablet_bootstrap.cc:654] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:40.895655 30666 log.cc:824] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:40.896199 30667 tablet_bootstrap.cc:654] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:40.897702 30667 log.cc:824] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:40.898303 30666 tablet_bootstrap.cc:492] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6: No bootstrap required, opened a new log
I20260812 03:13:40.898543 30666 ts_tablet_manager.cc:1397] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 03:13:40.900015 30667 tablet_bootstrap.cc:492] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2: No bootstrap required, opened a new log
I20260812 03:13:40.900254 30667 ts_tablet_manager.cc:1397] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 03:13:40.901124 30671 tablet_bootstrap.cc:492] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863: Bootstrap starting.
I20260812 03:13:40.903785 30671 tablet_bootstrap.cc:654] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:40.904780 30671 log.cc:824] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:40.904505 30666 raft_consensus.cc:348] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } }
I20260812 03:13:40.904927 30666 raft_consensus.cc:374] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:40.905002 30666 raft_consensus.cc:729] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f31718da75b84535bd70312147ab2aa6, State: Initialized, Role: FOLLOWER
I20260812 03:13:40.905598 30666 consensus_queue.cc:260] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } }
I20260812 03:13:40.906867 30666 ts_tablet_manager.cc:1428] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260812 03:13:40.906723 30667 raft_consensus.cc:348] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } }
I20260812 03:13:40.906987 30667 raft_consensus.cc:374] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:40.907066 30667 raft_consensus.cc:729] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f053a481a444a4a81364a9794f487c2, State: Initialized, Role: FOLLOWER
I20260812 03:13:40.907593 30667 consensus_queue.cc:260] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } }
I20260812 03:13:40.907814 30671 tablet_bootstrap.cc:492] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863: No bootstrap required, opened a new log
I20260812 03:13:40.908033 30671 ts_tablet_manager.cc:1397] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260812 03:13:40.908509 30604 heartbeater.cc:503] Master 127.16.217.126:41251 was elected leader, sending a full tablet report...
I20260812 03:13:40.908834 30667 ts_tablet_manager.cc:1428] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260812 03:13:40.916047 30671 raft_consensus.cc:348] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } }
I20260812 03:13:40.916363 30671 raft_consensus.cc:374] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:40.916462 30671 raft_consensus.cc:729] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91d973e36074da488958edfc2f5e863, State: Initialized, Role: FOLLOWER
I20260812 03:13:40.916941 30671 consensus_queue.cc:260] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } }
I20260812 03:13:40.917918 30376 heartbeater.cc:503] Master 127.16.217.126:41251 was elected leader, sending a full tablet report...
I20260812 03:13:40.918596 30671 ts_tablet_manager.cc:1428] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863: Time spent starting tablet: real 0.010s user 0.006s sys 0.005s
I20260812 03:13:40.919307 30136 heartbeater.cc:503] Master 127.16.217.126:41251 was elected leader, sending a full tablet report...
I20260812 03:13:40.923590 30674 raft_consensus.cc:482] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:40.923853 30674 raft_consensus.cc:504] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } }
I20260812 03:13:40.925560 30674 leader_election.cc:288] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e91d973e36074da488958edfc2f5e863 (127.16.217.66:39739), 4f053a481a444a4a81364a9794f487c2 (127.16.217.67:37525)
I20260812 03:13:40.930485 30661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.926312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:45979 (local address 127.16.217.67:37525)
0812 03:13:40.926441 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.926444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.926455 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:13:40.926498 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.926506 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.926546 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.926606 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:40.927452 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.928083 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.929411 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.929793 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.929887 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.929970 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.930291 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.930376 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:13:40.930459 30663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.926285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:33807 (local address 127.16.217.66:39739)
0812 03:13:40.926418 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.926425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.926440 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:13:40.926666 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.926676 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.926776 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.927075 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:40.927477 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.928112 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.929335 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.929826 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.929978 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.930059 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.930224 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.930331 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:13:40.930475 30679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.925775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:39739 (local address 127.16.217.65:33807)
0812 03:13:40.926123 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.926143 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.926276 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.927097 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.927100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.927118 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:40.927242 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.927247 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.928247 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.928250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.929204 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.929214 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.929994 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.929999 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.930091 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.930112 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.930176 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.930294 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":40,"thread_start_us":159,"threads_started":1}
I20260812 03:13:40.931005 30680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.925864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:37525 (local address 127.16.217.65:45979)
0812 03:13:40.926215 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:40.926253 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:40.926351 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:40.926765 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:40.926769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:40.926798 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:40.927157 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.927169 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.928196 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.928202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.929253 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:40.929292 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.929932 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.929938 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:40.930123 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.930145 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:13:40.930274 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 03:13:40.930341 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":5,"thread_start_us":138,"threads_started":1}
I20260812 03:13:40.931190 30288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "545396670dc34bf293d0c6cdd3416807" candidate_uuid: "f31718da75b84535bd70312147ab2aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91d973e36074da488958edfc2f5e863" is_pre_election: true
I20260812 03:13:40.931634 30288 raft_consensus.cc:2454] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f31718da75b84535bd70312147ab2aa6 in term 0.
I20260812 03:13:40.931926 30535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "545396670dc34bf293d0c6cdd3416807" candidate_uuid: "f31718da75b84535bd70312147ab2aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f053a481a444a4a81364a9794f487c2" is_pre_election: true
I20260812 03:13:40.932322 30535 raft_consensus.cc:2454] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f31718da75b84535bd70312147ab2aa6 in term 0.
I20260812 03:13:40.932303 29936 leader_election.cc:302] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e91d973e36074da488958edfc2f5e863, f31718da75b84535bd70312147ab2aa6; no voters:
I20260812 03:13:40.932849 30677 raft_consensus.cc:2781] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:13:40.933032 30677 raft_consensus.cc:482] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:13:40.933346 30677 raft_consensus.cc:3037] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:40.935436 30677 raft_consensus.cc:504] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } }
I20260812 03:13:40.935860 30677 leader_election.cc:288] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [CANDIDATE]: Term 1 election: Requested vote from peers e91d973e36074da488958edfc2f5e863 (127.16.217.66:39739), 4f053a481a444a4a81364a9794f487c2 (127.16.217.67:37525)
I20260812 03:13:40.936707 30288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "545396670dc34bf293d0c6cdd3416807" candidate_uuid: "f31718da75b84535bd70312147ab2aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91d973e36074da488958edfc2f5e863"
I20260812 03:13:40.936707 30535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "545396670dc34bf293d0c6cdd3416807" candidate_uuid: "f31718da75b84535bd70312147ab2aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f053a481a444a4a81364a9794f487c2"
I20260812 03:13:40.936858 30288 raft_consensus.cc:3037] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:40.936858 30535 raft_consensus.cc:3037] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:40.939038 30288 raft_consensus.cc:2454] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f31718da75b84535bd70312147ab2aa6 in term 1.
I20260812 03:13:40.939038 30535 raft_consensus.cc:2454] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f31718da75b84535bd70312147ab2aa6 in term 1.
I20260812 03:13:40.939358 29935 leader_election.cc:302] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f053a481a444a4a81364a9794f487c2, f31718da75b84535bd70312147ab2aa6; no voters:
I20260812 03:13:40.939601 30677 raft_consensus.cc:2781] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:40.939734 30677 raft_consensus.cc:686] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [term 1 LEADER]: Becoming Leader. State: Replica: f31718da75b84535bd70312147ab2aa6, State: Running, Role: LEADER
I20260812 03:13:40.940166 30677 consensus_queue.cc:237] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } }
I20260812 03:13:40.944173 28461 catalog_manager.cc:5495] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 reported cstate change: term changed from 0 to 1, leader changed from <none> to f31718da75b84535bd70312147ab2aa6 (127.16.217.65). New cstate: current_term: 1 leader_uuid: "f31718da75b84535bd70312147ab2aa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } health_report { overall_health: UNKNOWN } } }
I20260812 03:13:40.965929 30657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:40.961573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51612 (local address 127.16.217.65:34213)
0812 03:13:40.961757 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:40.961761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:40.961775 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:13:40.961852 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:40.961854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:40.961935 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:40.962066 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:40.962459 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.963276 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.964173 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:40.964484 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:40.964598 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:40.964955 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:40.965427 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:40.965617 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:40.965665 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 03:13:40.965728 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 03:13:40.973804 30535 raft_consensus.cc:1261] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2 [term 1 FOLLOWER]: Refusing update from remote peer f31718da75b84535bd70312147ab2aa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:40.973827 30288 raft_consensus.cc:1261] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863 [term 1 FOLLOWER]: Refusing update from remote peer f31718da75b84535bd70312147ab2aa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:40.974646 30677 consensus_queue.cc:1035] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:40.974885 30674 consensus_queue.cc:1035] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:41.001152 30690 mvcc.cc:205] Tried to move back new op lower bound from 7317522108300365824 to 7317522108173942784. Current Snapshot: MvccSnapshot[applied={T|T < 7317522108300365824 or (T in {7317522108300365824})}]
I20260812 03:13:41.002419 28461 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:41.002621 28461 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:41.007607 28461 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63dc7a5750d74d18ba2e2a6b6d38e6c5: Sending DeleteTablet for 3 replicas of tablet 545396670dc34bf293d0c6cdd3416807
I20260812 03:13:41.008992 30498 tablet_service.cc:1514] Processing DeleteTablet for tablet 545396670dc34bf293d0c6cdd3416807 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:41 UTC) from {username='oryx', principal='oryx/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:50540
I20260812 03:13:41.009075 30020 tablet_service.cc:1514] Processing DeleteTablet for tablet 545396670dc34bf293d0c6cdd3416807 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:41 UTC) from {username='oryx', principal='oryx/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:51606
I20260812 03:13:41.009378 30254 tablet_service.cc:1514] Processing DeleteTablet for tablet 545396670dc34bf293d0c6cdd3416807 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:41 UTC) from {username='oryx', principal='oryx/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:38178
I20260812 03:13:41.010002 30703 tablet_replica.cc:330] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863: stopping tablet replica
I20260812 03:13:41.010546 30703 raft_consensus.cc:2229] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:41.010578 30701 tablet_replica.cc:330] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6: stopping tablet replica
I20260812 03:13:41.010949 30703 raft_consensus.cc:2258] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:41.010954 30701 raft_consensus.cc:2229] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:13:41.011408 30701 raft_consensus.cc:2258] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:41.012099 30702 tablet_replica.cc:330] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2: stopping tablet replica
I20260812 03:13:41.012454 30702 raft_consensus.cc:2229] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:41.012835 30702 raft_consensus.cc:2258] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:41.012923 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 28406
I20260812 03:13:41.022641 30701 ts_tablet_manager.cc:1905] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:41.028371 30703 ts_tablet_manager.cc:1905] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:41.033618 30703 ts_tablet_manager.cc:1918] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:13:41.035004 30701 ts_tablet_manager.cc:1918] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:13:41.035168 30701 log.cc:1196] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-0/wal/wals/545396670dc34bf293d0c6cdd3416807
I20260812 03:13:41.035545 30701 ts_tablet_manager.cc:1939] T 545396670dc34bf293d0c6cdd3416807 P f31718da75b84535bd70312147ab2aa6: Deleting consensus metadata
I20260812 03:13:41.036969 30703 log.cc:1196] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/545396670dc34bf293d0c6cdd3416807
I20260812 03:13:41.037648 30703 ts_tablet_manager.cc:1939] T 545396670dc34bf293d0c6cdd3416807 P e91d973e36074da488958edfc2f5e863: Deleting consensus metadata
I20260812 03:13:41.041851 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:41.011362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48414 (local address 127.16.217.125:38415)
0812 03:13:41.011478 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:41.011485 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:41.027590 (+ 16105us) server_negotiation.cc:407] Connection header received
0812 03:13:41.027862 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:41.027868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:41.027927 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:41.027981 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:41.029167 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:41.029858 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:41.031020 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:41.031365 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:41.031448 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:41.037519 (+ 6071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:41.037524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:41.037526 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:41.037561 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:41.039521 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:41.040074 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:41.040081 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:41.040086 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:41.040205 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:41.040663 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:41.040669 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:41.040673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:41.040975 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:41.041188 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:41.041416 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 03:13:41.041589 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 03:13:41.044389 30176 connection.cc:535] server connection from 127.0.0.1:38178 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 03:13:41.045414 29936 connection.cc:535] server connection from 127.0.0.1:51606 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 03:13:41.049516 30702 ts_tablet_manager.cc:1905] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:41.052702 30145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:41.011565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49216 (local address 127.16.217.124:35743)
0812 03:13:41.011667 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:41.011671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:41.042936 (+ 31265us) server_negotiation.cc:407] Connection header received
0812 03:13:41.043144 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:41.043148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:41.043204 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:41.043256 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:41.044163 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:41.044936 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:41.046095 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:41.046464 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:41.046530 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:41.048053 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:41.048057 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:41.048059 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:41.048095 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:41.049955 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:41.050635 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:41.050640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:41.050642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:41.050750 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:41.051192 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:41.051196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:41.051199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:41.051448 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:41.051596 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:41.052348 (+ 752us) server_negotiation.cc:299] Negotiation successful
0812 03:13:41.052521 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:13:41.053189 30702 ts_tablet_manager.cc:1918] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:13:41.053345 30702 log.cc:1196] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/545396670dc34bf293d0c6cdd3416807
I20260812 03:13:41.053747 30702 ts_tablet_manager.cc:1939] T 545396670dc34bf293d0c6cdd3416807 P 4f053a481a444a4a81364a9794f487c2: Deleting consensus metadata
W20260812 03:13:41.055215 30414 connection.cc:441] server connection from 127.0.0.1:50540 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50540 (request call id 1) could send its response
I20260812 03:13:41.059399 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:41.048578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48428 (local address 127.16.217.125:38415)
0812 03:13:41.048719 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:41.048725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:41.049371 (+ 646us) server_negotiation.cc:407] Connection header received
0812 03:13:41.049571 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:41.049575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:41.049628 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:41.049686 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:41.051601 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:41.052340 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:41.053427 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:41.053751 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:41.053836 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:41.055852 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:41.055856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:41.055858 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:41.055890 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:41.057335 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:41.057787 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:41.057791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:41.057793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:41.057913 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:41.058316 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:41.058319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:41.058321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:41.058534 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:41.058702 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:41.059061 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 03:13:41.059198 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:13:41.067222 30629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:41.050476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49220 (local address 127.16.217.124:35743)
0812 03:13:41.050596 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:41.050600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:41.058642 (+ 8042us) server_negotiation.cc:407] Connection header received
0812 03:13:41.058789 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:41.058792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:41.058888 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:41.058969 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:41.059867 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:41.060581 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:41.061523 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:41.061768 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:41.061903 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:41.063647 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:41.063651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:41.063654 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:41.063686 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:41.065391 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:41.065846 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:41.065850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:41.065852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:41.065947 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:41.066317 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:41.066320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:41.066322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:41.066537 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:41.066667 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:41.066903 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 03:13:41.067043 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:13:41.988070 30714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:41.987536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.66:56169)
0812 03:13:41.987854 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:41.987953 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":85,"threads_started":1}
W20260812 03:13:41.988642 30376 heartbeater.cc:644] Failed to heartbeat to 127.16.217.126:41251 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
I20260812 03:13:41.995697 30715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:41.995113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.65:54971)
0812 03:13:41.995422 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:41.995569 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":120,"threads_started":1}
W20260812 03:13:41.996106 30136 heartbeater.cc:644] Failed to heartbeat to 127.16.217.126:41251 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
I20260812 03:13:41.998641 30717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:41.997946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.67:44353)
0812 03:13:41.998457 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:41.998549 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":83,"threads_started":1}
W20260812 03:13:41.998965 30604 heartbeater.cc:644] Failed to heartbeat to 127.16.217.126:41251 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
I20260812 03:13:42.507954 30721 raft_consensus.cc:482] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 63dc7a5750d74d18ba2e2a6b6d38e6c5)
I20260812 03:13:42.508119 30721 raft_consensus.cc:504] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: false } }
I20260812 03:13:42.508836 30721 leader_election.cc:288] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 63dc7a5750d74d18ba2e2a6b6d38e6c5 (127.16.217.126:41251), 82c62265ebbe4182a856f2f8759fd5ef (127.16.217.125:38415)
I20260812 03:13:42.508883 30722 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 63dc7a5750d74d18ba2e2a6b6d38e6c5)
I20260812 03:13:42.509217 30722 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: false } }
I20260812 03:13:42.509696 30723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.509136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.0.0.1:37316)
0812 03:13:42.509472 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.509571 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"spinlock_wait_cycles":1280,"thread_start_us":85,"threads_started":1}
W20260812 03:13:42.509891 29810 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
I20260812 03:13:42.510816 30723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.510551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.0.0.1:37320)
0812 03:13:42.510677 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.510744 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 03:13:42.510808 30722 leader_election.cc:288] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 63dc7a5750d74d18ba2e2a6b6d38e6c5 (127.16.217.126:41251), afac16e6c29649e5a414afb3b99f3c72 (127.16.217.124:35743)
I20260812 03:13:42.511301 30731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.510869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.0.0.1:37326)
0812 03:13:42.511094 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.511157 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":77,"threads_started":1}
W20260812 03:13:42.511473 29486 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
W20260812 03:13:42.511515 29810 leader_election.cc:334] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 63dc7a5750d74d18ba2e2a6b6d38e6c5 (127.16.217.126:41251): Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
I20260812 03:13:42.512418 30732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.512148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.0.0.1:37336)
0812 03:13:42.512284 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.512345 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 03:13:42.512570 29486 leader_election.cc:334] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 63dc7a5750d74d18ba2e2a6b6d38e6c5 (127.16.217.126:41251): Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
I20260812 03:13:42.513926 30729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.509184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48430 (local address 127.16.217.125:38415)
0812 03:13:42.509683 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:42.509689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:42.509709 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:13:42.509767 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:42.509770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:42.509824 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:42.509934 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:42.510305 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.511266 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.512861 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.513312 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.513458 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:42.513520 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:42.513705 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 03:13:42.513804 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":55,"threads_started":1}
I20260812 03:13:42.513948 30727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.509124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38415 (local address 127.0.0.1:48430)
0812 03:13:42.509472 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.509488 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:42.509613 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:42.509948 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:42.509952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:42.510022 (+ 70us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:42.510144 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:42.510152 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.511413 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.511418 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:42.512708 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:42.512723 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.513448 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.513452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:42.513553 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:42.513580 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 03:13:42.513688 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 03:13:42.513767 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":81,"threads_started":1}
I20260812 03:13:42.514390 29526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "afac16e6c29649e5a414afb3b99f3c72" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "82c62265ebbe4182a856f2f8759fd5ef" is_pre_election: true
I20260812 03:13:42.514560 29526 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afac16e6c29649e5a414afb3b99f3c72 in term 1.
I20260812 03:13:42.514935 29807 leader_election.cc:302] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 82c62265ebbe4182a856f2f8759fd5ef, afac16e6c29649e5a414afb3b99f3c72; no voters: 63dc7a5750d74d18ba2e2a6b6d38e6c5
I20260812 03:13:42.515157 30721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 03:13:42.515224 30721 raft_consensus.cc:482] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 63dc7a5750d74d18ba2e2a6b6d38e6c5)
I20260812 03:13:42.515311 30721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:13:42.515332 30731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.511226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35743 (local address 127.0.0.1:49226)
0812 03:13:42.511461 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.511476 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:42.511571 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:42.512140 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:42.512143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:42.512165 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:42.512254 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:42.512260 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.513115 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.513120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:42.513960 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:42.513969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.514916 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.514920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:42.515013 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:42.515034 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 03:13:42.515130 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 03:13:42.515200 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":61,"threads_started":1}
I20260812 03:13:42.515713 30734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.511644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49226 (local address 127.16.217.124:35743)
0812 03:13:42.511903 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:42.511914 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:42.511930 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:13:42.511984 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:42.511987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:42.512031 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:42.512126 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:42.512380 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.513004 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.514143 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.514798 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.514922 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:42.514966 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:42.515573 (+ 607us) server_negotiation.cc:299] Negotiation successful
0812 03:13:42.515634 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":90,"threads_started":1}
I20260812 03:13:42.516148 29852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "82c62265ebbe4182a856f2f8759fd5ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "afac16e6c29649e5a414afb3b99f3c72" is_pre_election: true
I20260812 03:13:42.516672 30721 raft_consensus.cc:504] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: false } }
I20260812 03:13:42.517007 29852 raft_consensus.cc:2379] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 82c62265ebbe4182a856f2f8759fd5ef in current term 2: Already voted for candidate afac16e6c29649e5a414afb3b99f3c72 in this term.
I20260812 03:13:42.517109 30721 leader_election.cc:288] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [CANDIDATE]: Term 2 election: Requested vote from peers 63dc7a5750d74d18ba2e2a6b6d38e6c5 (127.16.217.126:41251), 82c62265ebbe4182a856f2f8759fd5ef (127.16.217.125:38415)
I20260812 03:13:42.517468 30727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.517211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.0.0.1:37344)
0812 03:13:42.517310 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.517409 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 03:13:42.517547 29481 leader_election.cc:302] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 82c62265ebbe4182a856f2f8759fd5ef; no voters: 63dc7a5750d74d18ba2e2a6b6d38e6c5, afac16e6c29649e5a414afb3b99f3c72
I20260812 03:13:42.517760 30722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:13:42.517829 29526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "afac16e6c29649e5a414afb3b99f3c72" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "82c62265ebbe4182a856f2f8759fd5ef"
I20260812 03:13:42.518401 30727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.518146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.0.0.1:37358)
0812 03:13:42.518291 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.518351 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 03:13:42.518510 29810 leader_election.cc:334] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 63dc7a5750d74d18ba2e2a6b6d38e6c5 (127.16.217.126:41251): Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
I20260812 03:13:42.518790 30722 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:13:42.519821 29526 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afac16e6c29649e5a414afb3b99f3c72 in term 2.
I20260812 03:13:42.520896 29807 leader_election.cc:302] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 82c62265ebbe4182a856f2f8759fd5ef, afac16e6c29649e5a414afb3b99f3c72; no voters: 63dc7a5750d74d18ba2e2a6b6d38e6c5
I20260812 03:13:42.521095 30721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:13:42.521334 30721 raft_consensus.cc:686] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [term 2 LEADER]: Becoming Leader. State: Replica: afac16e6c29649e5a414afb3b99f3c72, State: Running, Role: LEADER
I20260812 03:13:42.521615 30721 consensus_queue.cc:237] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: false } }
I20260812 03:13:42.522835 30737 sys_catalog.cc:455] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afac16e6c29649e5a414afb3b99f3c72. Latest consensus state: current_term: 2 leader_uuid: "afac16e6c29649e5a414afb3b99f3c72" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: false } } }
I20260812 03:13:42.522964 30737 sys_catalog.cc:458] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:42.523286 30740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:42.525882 30740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:42.526403 30740 catalog_manager.cc:1259] Loaded cluster ID: 194f5c0e6bc648f3b560b57996f4cddd
I20260812 03:13:42.526484 30740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:42.527077 30740 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:42.527544 30740 catalog_manager.cc:5879] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: Loaded TSK: 0
I20260812 03:13:42.528321 30740 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:13:42.540743 29839 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:13:42.543987 29839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:13:42.547356 30727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.546953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.0.0.1:37370)
0812 03:13:42.547154 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.547215 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 03:13:42.547492 29526 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [term 2 FOLLOWER]: Refusing update from remote peer afac16e6c29649e5a414afb3b99f3c72: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 03:13:42.548185 30727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.547932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.0.0.1:37374)
0812 03:13:42.548059 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.548119 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 03:13:42.548194 30737 consensus_queue.cc:1035] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 03:13:42.548405 29810 consensus_peers.cc:487] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 -> Peer 63dc7a5750d74d18ba2e2a6b6d38e6c5 (127.16.217.126:41251): Couldn't send request to peer 63dc7a5750d74d18ba2e2a6b6d38e6c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 03:13:42.549955 30722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader afac16e6c29649e5a414afb3b99f3c72. Latest consensus state: current_term: 2 leader_uuid: "afac16e6c29649e5a414afb3b99f3c72" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: false } } }
I20260812 03:13:42.550139 30722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:13:42.551301 30737 sys_catalog.cc:455] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "afac16e6c29649e5a414afb3b99f3c72" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: false } } }
I20260812 03:13:42.551427 30737 sys_catalog.cc:458] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:42.551832 30722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "afac16e6c29649e5a414afb3b99f3c72" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "63dc7a5750d74d18ba2e2a6b6d38e6c5" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 41251 } } peers { permanent_uuid: "82c62265ebbe4182a856f2f8759fd5ef" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38415 } attrs { promote: false } } peers { permanent_uuid: "afac16e6c29649e5a414afb3b99f3c72" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35743 } attrs { promote: false } } }
I20260812 03:13:42.551944 30722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c62265ebbe4182a856f2f8759fd5ef [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:13:42.562057 30727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.557887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:34213 (local address 127.0.0.1:51614)
0812 03:13:42.558068 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.558084 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:42.558171 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:42.558633 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:42.558636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:42.558653 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:42.558798 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:42.558806 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.559858 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.559864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:42.560628 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:42.560660 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.561665 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.561669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:42.561780 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:42.561811 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 03:13:42.561861 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 03:13:42.561926 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260812 03:13:42.562727 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.557987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51614 (local address 127.16.217.65:34213)
0812 03:13:42.558368 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:42.558372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:42.558386 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:13:42.558441 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:42.558445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:42.558494 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:42.558919 (+ 425us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:42.559044 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.559750 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.560760 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.561512 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.562391 (+ 879us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:42.562460 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:42.562495 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 03:13:42.562571 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":140,"threads_started":1}
I20260812 03:13:42.563304 30723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.558421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:39739 (local address 127.0.0.1:38186)
0812 03:13:42.558523 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.558539 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:42.558629 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:42.559413 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:42.559416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:42.559433 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:42.559550 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:42.559558 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.560702 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.560708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:42.561949 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:42.561961 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.562916 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.562919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:42.563015 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:42.563034 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 03:13:42.563075 (+ 41us) client_negotiation.cc:240] Negotiation successful
0812 03:13:42.563177 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":25}
I20260812 03:13:42.563233 30020 tablet_service.cc:1467] Processing CreateTablet for tablet 1c61530319f84d37b8280266f93aebf6 (DEFAULT_TABLE table=test-table [id=328142d08d0f45fd9f342690c28c08b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:42.563567 30020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c61530319f84d37b8280266f93aebf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:42.563571 30761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.558902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50548 (local address 127.16.217.67:37525)
0812 03:13:42.559282 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:42.559287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:42.559305 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:13:42.559364 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:42.559368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:42.559422 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:42.559525 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:42.559773 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.560574 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.562405 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.563026 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.563136 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:42.563207 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:42.563362 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 03:13:42.563433 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":78,"threads_started":1}
I20260812 03:13:42.563902 30754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.558559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:37525 (local address 127.0.0.1:50548)
0812 03:13:42.559080 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.559094 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:42.559194 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:42.559547 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:42.559549 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:42.559573 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:42.559651 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:42.559656 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.560731 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.560736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:42.562264 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:42.562275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.563150 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.563153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:42.563230 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:42.563251 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 03:13:42.563333 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:13:42.563582 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":23,"thread_start_us":109,"threads_started":1}
I20260812 03:13:42.563901 30759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.558868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38186 (local address 127.16.217.66:39739)
0812 03:13:42.559132 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:42.559138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:42.559155 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:13:42.559215 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:42.559218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:42.559270 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:42.559664 (+ 394us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:42.559807 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.560574 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.562122 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.562745 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.563589 (+ 844us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:42.563658 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:42.563704 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 03:13:42.563771 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":100,"threads_started":1}
I20260812 03:13:42.564411 30254 tablet_service.cc:1467] Processing CreateTablet for tablet 1c61530319f84d37b8280266f93aebf6 (DEFAULT_TABLE table=test-table [id=328142d08d0f45fd9f342690c28c08b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:42.564666 30254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c61530319f84d37b8280266f93aebf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:42.564739 30498 tablet_service.cc:1467] Processing CreateTablet for tablet 1c61530319f84d37b8280266f93aebf6 (DEFAULT_TABLE table=test-table [id=328142d08d0f45fd9f342690c28c08b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:42.565061 30498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c61530319f84d37b8280266f93aebf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:42.566457 30762 tablet_bootstrap.cc:492] T 1c61530319f84d37b8280266f93aebf6 P f31718da75b84535bd70312147ab2aa6: Bootstrap starting.
I20260812 03:13:42.567329 30763 tablet_bootstrap.cc:492] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863: Bootstrap starting.
I20260812 03:13:42.567719 30764 tablet_bootstrap.cc:492] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2: Bootstrap starting.
I20260812 03:13:42.567652 30762 tablet_bootstrap.cc:654] T 1c61530319f84d37b8280266f93aebf6 P f31718da75b84535bd70312147ab2aa6: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:42.568303 30763 tablet_bootstrap.cc:654] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:42.568701 30764 tablet_bootstrap.cc:654] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:42.569108 30762 tablet_bootstrap.cc:492] T 1c61530319f84d37b8280266f93aebf6 P f31718da75b84535bd70312147ab2aa6: No bootstrap required, opened a new log
I20260812 03:13:42.569242 30762 ts_tablet_manager.cc:1397] T 1c61530319f84d37b8280266f93aebf6 P f31718da75b84535bd70312147ab2aa6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:42.569785 30762 raft_consensus.cc:348] T 1c61530319f84d37b8280266f93aebf6 P f31718da75b84535bd70312147ab2aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } }
I20260812 03:13:42.569979 30762 raft_consensus.cc:374] T 1c61530319f84d37b8280266f93aebf6 P f31718da75b84535bd70312147ab2aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:42.570024 30762 raft_consensus.cc:729] T 1c61530319f84d37b8280266f93aebf6 P f31718da75b84535bd70312147ab2aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f31718da75b84535bd70312147ab2aa6, State: Initialized, Role: FOLLOWER
I20260812 03:13:42.570181 30762 consensus_queue.cc:260] T 1c61530319f84d37b8280266f93aebf6 P f31718da75b84535bd70312147ab2aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } }
I20260812 03:13:42.570434 30764 tablet_bootstrap.cc:492] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2: No bootstrap required, opened a new log
I20260812 03:13:42.570514 30763 tablet_bootstrap.cc:492] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863: No bootstrap required, opened a new log
I20260812 03:13:42.570600 30764 ts_tablet_manager.cc:1397] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:42.570802 30762 ts_tablet_manager.cc:1428] T 1c61530319f84d37b8280266f93aebf6 P f31718da75b84535bd70312147ab2aa6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:13:42.571177 30764 raft_consensus.cc:348] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } }
I20260812 03:13:42.571352 30764 raft_consensus.cc:374] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:42.571447 30764 raft_consensus.cc:729] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f053a481a444a4a81364a9794f487c2, State: Initialized, Role: FOLLOWER
I20260812 03:13:42.571604 30764 consensus_queue.cc:260] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } }
I20260812 03:13:42.571980 30764 ts_tablet_manager.cc:1428] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:42.573480 30602 heartbeater.cc:503] Master 127.16.217.124:35743 was elected leader, sending a full tablet report...
W20260812 03:13:42.573783 30607 tablet.cc:2369] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:13:42.573985 30134 heartbeater.cc:503] Master 127.16.217.124:35743 was elected leader, sending a full tablet report...
I20260812 03:13:42.573909 30763 ts_tablet_manager.cc:1397] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.001s
I20260812 03:13:42.575109 30763 raft_consensus.cc:348] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } }
I20260812 03:13:42.575232 30763 raft_consensus.cc:374] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:42.575290 30763 raft_consensus.cc:729] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91d973e36074da488958edfc2f5e863, State: Initialized, Role: FOLLOWER
I20260812 03:13:42.575433 30763 consensus_queue.cc:260] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } }
I20260812 03:13:42.575817 30763 ts_tablet_manager.cc:1428] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:13:42.576256 30773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.573458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.67:58585)
0812 03:13:42.576079 (+ 2621us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.576152 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2548,"thread_start_us":126,"threads_started":1}
I20260812 03:13:42.576673 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.570939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.65:33625)
0812 03:13:42.576515 (+ 5576us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.576574 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5502,"thread_start_us":120,"threads_started":1}
I20260812 03:13:42.577353 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.577136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.65:40881)
0812 03:13:42.577243 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.577291 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 03:13:42.577646 30374 heartbeater.cc:503] Master 127.16.217.124:35743 was elected leader, sending a full tablet report...
I20260812 03:13:42.577756 30773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.577493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.67:43649)
0812 03:13:42.577641 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.577692 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 03:13:42.578065 30776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.576532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.66:33663)
0812 03:13:42.577121 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.577957 (+ 836us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":99,"threads_started":1}
I20260812 03:13:42.578753 30776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.578501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.66:38597)
0812 03:13:42.578620 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.578667 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 03:13:42.592760 30141 tablet.cc:2369] T 1c61530319f84d37b8280266f93aebf6 P f31718da75b84535bd70312147ab2aa6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:13:42.595216 30380 tablet.cc:2369] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:13:42.838910 30772 raft_consensus.cc:482] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:42.839197 30772 raft_consensus.cc:504] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } }
I20260812 03:13:42.841002 30772 leader_election.cc:288] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f31718da75b84535bd70312147ab2aa6 (127.16.217.65:34213), e91d973e36074da488958edfc2f5e863 (127.16.217.66:39739)
I20260812 03:13:42.845883 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.841502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:53979 (local address 127.16.217.65:34213)
0812 03:13:42.841648 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:42.841652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:42.841668 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:13:42.841890 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:42.841893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:42.841935 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:42.842023 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:42.842664 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.843543 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.844793 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.845300 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.845433 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:42.845492 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:42.845679 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 03:13:42.845753 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:13:42.845883 30773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.841348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:34213 (local address 127.16.217.67:53979)
0812 03:13:42.841524 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.841544 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:42.841691 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:42.842370 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:42.842374 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:42.842396 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:42.842536 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:42.842542 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.843707 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.843711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:42.844627 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:42.844637 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.845433 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.845437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:42.845549 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:42.845569 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 03:13:42.845659 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 03:13:42.845744 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260812 03:13:42.846674 30056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c61530319f84d37b8280266f93aebf6" candidate_uuid: "4f053a481a444a4a81364a9794f487c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f31718da75b84535bd70312147ab2aa6" is_pre_election: true
I20260812 03:13:42.846959 30056 raft_consensus.cc:2454] T 1c61530319f84d37b8280266f93aebf6 P f31718da75b84535bd70312147ab2aa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f053a481a444a4a81364a9794f487c2 in term 0.
I20260812 03:13:42.847096 30778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.841348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:39739 (local address 127.16.217.67:44717)
0812 03:13:42.842499 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.842512 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:42.842666 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:42.842945 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:42.842948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:42.842965 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:42.843084 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:42.843090 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.844332 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.844339 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:42.845445 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:42.845455 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.846648 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.846652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:42.846765 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:42.846789 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:13:42.846872 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 03:13:42.846936 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"mutex_wait_us":191,"thread_start_us":190,"threads_started":1}
I20260812 03:13:42.847370 30759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.841486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:44717 (local address 127.16.217.66:39739)
0812 03:13:42.841636 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:42.841644 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:42.842621 (+ 977us) server_negotiation.cc:407] Connection header received
0812 03:13:42.842788 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:42.842791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:42.842841 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:42.842922 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:42.843199 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.844195 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.845570 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.846228 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.846691 (+ 463us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:42.846752 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:42.846907 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 03:13:42.846995 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:13:42.847625 30415 leader_election.cc:302] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f053a481a444a4a81364a9794f487c2, f31718da75b84535bd70312147ab2aa6; no voters:
I20260812 03:13:42.847954 30772 raft_consensus.cc:2781] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:13:42.848037 30772 raft_consensus.cc:482] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:13:42.847999 30288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c61530319f84d37b8280266f93aebf6" candidate_uuid: "4f053a481a444a4a81364a9794f487c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91d973e36074da488958edfc2f5e863" is_pre_election: true
I20260812 03:13:42.848086 30772 raft_consensus.cc:3037] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:42.848165 30288 raft_consensus.cc:2454] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f053a481a444a4a81364a9794f487c2 in term 0.
I20260812 03:13:42.849380 30772 raft_consensus.cc:504] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } }
I20260812 03:13:42.849761 30772 leader_election.cc:288] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [CANDIDATE]: Term 1 election: Requested vote from peers f31718da75b84535bd70312147ab2aa6 (127.16.217.65:34213), e91d973e36074da488958edfc2f5e863 (127.16.217.66:39739)
I20260812 03:13:42.850113 30056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c61530319f84d37b8280266f93aebf6" candidate_uuid: "4f053a481a444a4a81364a9794f487c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f31718da75b84535bd70312147ab2aa6"
I20260812 03:13:42.850168 30288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c61530319f84d37b8280266f93aebf6" candidate_uuid: "4f053a481a444a4a81364a9794f487c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91d973e36074da488958edfc2f5e863"
I20260812 03:13:42.850281 30056 raft_consensus.cc:3037] T 1c61530319f84d37b8280266f93aebf6 P f31718da75b84535bd70312147ab2aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:42.850324 30288 raft_consensus.cc:3037] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:42.851241 30288 raft_consensus.cc:2454] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f053a481a444a4a81364a9794f487c2 in term 1.
I20260812 03:13:42.851315 30056 raft_consensus.cc:2454] T 1c61530319f84d37b8280266f93aebf6 P f31718da75b84535bd70312147ab2aa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f053a481a444a4a81364a9794f487c2 in term 1.
I20260812 03:13:42.851536 30414 leader_election.cc:302] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f053a481a444a4a81364a9794f487c2, e91d973e36074da488958edfc2f5e863; no voters:
I20260812 03:13:42.851706 30772 raft_consensus.cc:2781] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:42.851979 30772 raft_consensus.cc:686] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [term 1 LEADER]: Becoming Leader. State: Replica: 4f053a481a444a4a81364a9794f487c2, State: Running, Role: LEADER
I20260812 03:13:42.852259 30772 consensus_queue.cc:237] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } }
I20260812 03:13:42.853121 30778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.852673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.67:37195)
0812 03:13:42.853006 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.853060 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266}
I20260812 03:13:42.855816 29838 catalog_manager.cc:5495] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f053a481a444a4a81364a9794f487c2 (127.16.217.67). New cstate: current_term: 1 leader_uuid: "4f053a481a444a4a81364a9794f487c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f053a481a444a4a81364a9794f487c2" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37525 } health_report { overall_health: HEALTHY } } }
I20260812 03:13:42.933207 30761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.929229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50560 (local address 127.16.217.67:37525)
0812 03:13:42.929438 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:42.929442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:42.929466 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:13:42.929546 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:42.929548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:42.929609 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:42.929718 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:42.929984 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.930799 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.931685 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:42.931915 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:42.932029 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:42.932275 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:42.932801 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:42.932906 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:42.933024 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 03:13:42.933084 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260812 03:13:42.936337 30288 raft_consensus.cc:1261] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863 [term 1 FOLLOWER]: Refusing update from remote peer 4f053a481a444a4a81364a9794f487c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:42.936564 30056 raft_consensus.cc:1261] T 1c61530319f84d37b8280266f93aebf6 P f31718da75b84535bd70312147ab2aa6 [term 1 FOLLOWER]: Refusing update from remote peer 4f053a481a444a4a81364a9794f487c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:42.936887 30772 consensus_queue.cc:1035] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91d973e36074da488958edfc2f5e863" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:42.937194 30779 consensus_queue.cc:1035] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f31718da75b84535bd70312147ab2aa6" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 34213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:42.938951 30776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.938622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.66:55481)
0812 03:13:42.938800 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.938867 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260812 03:13:42.941994 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.941678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.65:37793)
0812 03:13:42.941842 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.941906 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260812 03:13:42.943580 30778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.943101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.67:36635)
0812 03:13:42.943439 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.943497 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263}
I20260812 03:13:42.945417 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.944986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.65:47559)
0812 03:13:42.945146 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.945240 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260812 03:13:42.951890 30778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.951586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.67:37425)
0812 03:13:42.951744 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.951802 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260812 03:13:42.953009 30776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.952481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.16.217.66:33909)
0812 03:13:42.952856 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.952925 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300}
I20260812 03:13:42.954404 29838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:42.954568 29838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:42.959482 29838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P afac16e6c29649e5a414afb3b99f3c72: Sending DeleteTablet for 3 replicas of tablet 1c61530319f84d37b8280266f93aebf6
I20260812 03:13:42.960377 30498 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c61530319f84d37b8280266f93aebf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:42 UTC) from {username='oryx', principal='oryx/127.16.217.124@KRBTEST.COM'} at 127.0.0.1:50548
I20260812 03:13:42.961021 30254 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c61530319f84d37b8280266f93aebf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:42 UTC) from {username='oryx', principal='oryx/127.16.217.124@KRBTEST.COM'} at 127.0.0.1:38186
I20260812 03:13:42.961086 30020 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c61530319f84d37b8280266f93aebf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:42 UTC) from {username='oryx', principal='oryx/127.16.217.124@KRBTEST.COM'} at 127.0.0.1:51614
I20260812 03:13:42.961212 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 29743
I20260812 03:13:42.970615 30798 tablet_replica.cc:330] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863: stopping tablet replica
I20260812 03:13:42.970816 30798 raft_consensus.cc:2229] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:42.971001 30798 raft_consensus.cc:2258] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:42.971897 30798 ts_tablet_manager.cc:1905] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:42.975555 30798 ts_tablet_manager.cc:1918] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:13:42.975675 30798 log.cc:1196] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/1c61530319f84d37b8280266f93aebf6
I20260812 03:13:42.975994 30798 ts_tablet_manager.cc:1939] T 1c61530319f84d37b8280266f93aebf6 P e91d973e36074da488958edfc2f5e863: Deleting consensus metadata
I20260812 03:13:42.976744 29807 catalog_manager.cc:4841] TS e91d973e36074da488958edfc2f5e863 (127.16.217.66:39739): tablet 1c61530319f84d37b8280266f93aebf6 (table test-table [id=328142d08d0f45fd9f342690c28c08b8]) successfully deleted
I20260812 03:13:42.978492 30797 tablet_replica.cc:330] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2: stopping tablet replica
I20260812 03:13:42.978655 30797 raft_consensus.cc:2229] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:13:42.978981 30797 raft_consensus.cc:2258] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:42.979699 30797 ts_tablet_manager.cc:1905] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:42.981765 30797 ts_tablet_manager.cc:1918] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:13:42.981863 30797 log.cc:1196] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/1c61530319f84d37b8280266f93aebf6
I20260812 03:13:42.982151 30797 ts_tablet_manager.cc:1939] T 1c61530319f84d37b8280266f93aebf6 P 4f053a481a444a4a81364a9794f487c2: Deleting consensus metadata
I20260812 03:13:42.982903 29806 catalog_manager.cc:4841] TS 4f053a481a444a4a81364a9794f487c2 (127.16.217.67:37525): tablet 1c61530319f84d37b8280266f93aebf6 (table test-table [id=328142d08d0f45fd9f342690c28c08b8]) successfully deleted
I20260812 03:13:42.987072 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 30146
I20260812 03:13:42.987298 30754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:42.986894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:34213 (local address 127.0.0.1:51620)
0812 03:13:42.987130 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:42.987195 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:34213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260812 03:13:42.989912 29808 catalog_manager.cc:4584] TS f31718da75b84535bd70312147ab2aa6 (127.16.217.65:34213): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1c61530319f84d37b8280266f93aebf6: Network error: Client connection negotiation failed: client connection to 127.16.217.65:34213: connect: Connection refused (error 111)
I20260812 03:13:43.004315 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 30384
I20260812 03:13:43.014554 30754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:43.014223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.0.0.1:37388)
0812 03:13:43.014400 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:43.014469 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 03:13:43.015379 30754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:43.015129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:41251 (local address 127.0.0.1:37392)
0812 03:13:43.015283 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:43.015327 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:41251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 03:13:43.020125 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 29259
I20260812 03:13:43.035185 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 29453
2026-08-12T03:13:43Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6031 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:13:43 dist-test-slave-7nm7 krb5kdc[30804](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:13:43 dist-test-slave-7nm7 krb5kdc[30804](info): set up 2 sockets
Aug 12 03:13:43 dist-test-slave-7nm7 krb5kdc[30804](info): commencing operation
krb5kdc: starting...
W20260812 03:13:44.396270 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.331s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:13:44 dist-test-slave-7nm7 krb5kdc[30804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:13:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:44Z Disabled control of system clock
WARNING: no policy specified for oryx/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.126.keytab.
Entry for principal oryx/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.126.keytab.
I20260812 03:13:44.538669 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:39749
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.126.keytab
--principal=oryx/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:39749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:44.663713 30836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:44.664055 30836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:44.664155 30836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:44.668424 30836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:44.668516 30836 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 03:13:44.668546 30836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:44.668592 30836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:44.668632 30836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:44.668684 30836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:13:44.674256 30836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:39749
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.126.keytab
--principal=oryx/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:39749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:44.675498 30836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:44.676584 30836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:44.686097 30848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:44.686098 30846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:44.686445 30836 server_base.cc:1034] running on GCE node
W20260812 03:13:44.686097 30845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:44.687197 30836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:44.688472 30836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:44.689714 30836 hybrid_clock.cc:648] HybridClock initialized: now 1786504424689682 us; error 46 us; skew 500 ppm
Aug 12 03:13:44 dist-test-slave-7nm7 krb5kdc[30804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504424, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:44.693383 30836 init.cc:377] Logged in from keytab as oryx/127.16.217.126@KRBTEST.COM (short username oryx)
I20260812 03:13:44.694618 30836 webserver.cc:460] Webserver started at http://127.16.217.126:36577/ using document root <none> and password file <none>
I20260812 03:13:44.695310 30836 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:44.695410 30836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:44.695665 30836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:44.697883 30836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "86707fb71bee45e6b0a40a55f9543479"
format_stamp: "Formatted at 2026-08-12 03:13:44 on dist-test-slave-7nm7"
server_key: "2483dc9ab792ad386b16d7b806c16a52"
server_key_iv: "9f738acc5bdcd68b94f5285b230ca0c4"
server_key_version: "encryptionkey@0"
I20260812 03:13:44.698519 30836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "86707fb71bee45e6b0a40a55f9543479"
format_stamp: "Formatted at 2026-08-12 03:13:44 on dist-test-slave-7nm7"
server_key: "2483dc9ab792ad386b16d7b806c16a52"
server_key_iv: "9f738acc5bdcd68b94f5285b230ca0c4"
server_key_version: "encryptionkey@0"
I20260812 03:13:44.703172 30836 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:13:44.705878 30855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:44.707398 30836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:13:44.707568 30836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "86707fb71bee45e6b0a40a55f9543479"
format_stamp: "Formatted at 2026-08-12 03:13:44 on dist-test-slave-7nm7"
server_key: "2483dc9ab792ad386b16d7b806c16a52"
server_key_iv: "9f738acc5bdcd68b94f5285b230ca0c4"
server_key_version: "encryptionkey@0"
I20260812 03:13:44.707700 30836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:44.725123 30836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:44.728734 30836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:44.729063 30836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:44.738979 30836 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:39749
I20260812 03:13:44.738999 30964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:39749 every 8 connection(s)
I20260812 03:13:44.740000 30836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:44.743160 30965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:44.748373 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 30836
I20260812 03:13:44.748524 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:13:44.748837 17253 external_mini_cluster.cc:1428] Setting key 0ea9f6b09db88712413cfd922ceb4078
I20260812 03:13:44.749032 30965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479: Bootstrap starting.
I20260812 03:13:44.751649 30965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:44.752583 30965 log.cc:824] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:44.754786 30965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479: No bootstrap required, opened a new log
Aug 12 03:13:44 dist-test-slave-7nm7 krb5kdc[30804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM
I20260812 03:13:44.759476 30965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86707fb71bee45e6b0a40a55f9543479" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39749 } }
I20260812 03:13:44.759749 30965 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:44.759786 30965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86707fb71bee45e6b0a40a55f9543479, State: Initialized, Role: FOLLOWER
I20260812 03:13:44.760447 30965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86707fb71bee45e6b0a40a55f9543479" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39749 } }
I20260812 03:13:44.760627 30965 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:44.760731 30965 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:44.760844 30965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:44.761914 30965 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86707fb71bee45e6b0a40a55f9543479" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39749 } }
I20260812 03:13:44.762391 30965 leader_election.cc:302] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86707fb71bee45e6b0a40a55f9543479; no voters:
I20260812 03:13:44.762773 30965 leader_election.cc:288] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:13:44.762859 30974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:44.763053 30974 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479 [term 1 LEADER]: Becoming Leader. State: Replica: 86707fb71bee45e6b0a40a55f9543479, State: Running, Role: LEADER
I20260812 03:13:44.763360 30974 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86707fb71bee45e6b0a40a55f9543479" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39749 } }
I20260812 03:13:44.764156 30965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:44.765336 30976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86707fb71bee45e6b0a40a55f9543479. Latest consensus state: current_term: 1 leader_uuid: "86707fb71bee45e6b0a40a55f9543479" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86707fb71bee45e6b0a40a55f9543479" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39749 } } }
I20260812 03:13:44.765460 30976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:44.765818 30979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86707fb71bee45e6b0a40a55f9543479" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86707fb71bee45e6b0a40a55f9543479" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39749 } } }
I20260812 03:13:44.765916 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:44.750322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53498 (local address 127.16.217.126:39749)
0812 03:13:44.750802 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:44.750817 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:44.750852 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:13:44.751489 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:44.751506 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:44.751881 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:44.752218 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:44.753219 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:44.754356 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:44.755364 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:44.755844 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:44.756046 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:44.759491 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:44.759497 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:44.759513 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:44.759544 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:44.761664 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:44.762532 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:44.762539 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:44.762545 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:44.762721 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:44.763208 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:44.763211 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:44.763213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:44.763844 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:44.764117 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:44.764400 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 03:13:44.764770 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":154,"threads_started":1}
I20260812 03:13:44.766152 30988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:44.765916 30979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:44.769631 30988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:44.775719 30988 catalog_manager.cc:1347] Generated new cluster ID: c299e6cf7c2c40abb9ad138af4c9e16d
I20260812 03:13:44.775791 30988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:44.789271 30988 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:13:44.790935 30988 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:44.795346 30988 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479: Generated new TSK 0
I20260812 03:13:44.796355 30988 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.65.keytab.
Entry for principal oryx/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.65.keytab.
I20260812 03:13:44.842834 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39749
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.65.keytab
--principal=oryx/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:44.982928 31003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:44.983176 31003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:44.983220 31003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:44.987466 31003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:44.987534 31003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:44.987574 31003 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:13:44.987607 31003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:13:44.992806 31003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.65.keytab
--principal=oryx/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.217.126:39749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:44.993815 31003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:44.994843 31003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:45.004727 31016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:45.004768 31018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:45.004921 31003 server_base.cc:1034] running on GCE node
W20260812 03:13:45.004787 31020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:45.005470 31003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:45.006124 31003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:45.007298 31003 hybrid_clock.cc:648] HybridClock initialized: now 1786504425007279 us; error 30 us; skew 500 ppm
Aug 12 03:13:45 dist-test-slave-7nm7 krb5kdc[30804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504425, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:45.010573 31003 init.cc:377] Logged in from keytab as oryx/127.16.217.65@KRBTEST.COM (short username oryx)
I20260812 03:13:45.011659 31003 webserver.cc:460] Webserver started at http://127.16.217.65:39941/ using document root <none> and password file <none>
I20260812 03:13:45.012248 31003 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:45.012355 31003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:45.012611 31003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:45.014784 31003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "652b61c0fc3f41cf8294353cfb3c4613"
format_stamp: "Formatted at 2026-08-12 03:13:45 on dist-test-slave-7nm7"
server_key: "12d72c58256d2d8fe893b558aeddc7f8"
server_key_iv: "3ca52ad0464dd245bba3178b7f3f807a"
server_key_version: "encryptionkey@0"
I20260812 03:13:45.015365 31003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "652b61c0fc3f41cf8294353cfb3c4613"
format_stamp: "Formatted at 2026-08-12 03:13:45 on dist-test-slave-7nm7"
server_key: "12d72c58256d2d8fe893b558aeddc7f8"
server_key_iv: "3ca52ad0464dd245bba3178b7f3f807a"
server_key_version: "encryptionkey@0"
I20260812 03:13:45.019335 31003 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:13:45.021873 31039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:45.023242 31003 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:13:45.023406 31003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "652b61c0fc3f41cf8294353cfb3c4613"
format_stamp: "Formatted at 2026-08-12 03:13:45 on dist-test-slave-7nm7"
server_key: "12d72c58256d2d8fe893b558aeddc7f8"
server_key_iv: "3ca52ad0464dd245bba3178b7f3f807a"
server_key_version: "encryptionkey@0"
I20260812 03:13:45.023536 31003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:45.051999 31003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:45.055289 31003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:45.055521 31003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:45.057242 31003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:45.057331 31003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:45.057435 31003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:45.057487 31003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:13:45 dist-test-slave-7nm7 krb5kdc[30804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504425, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM
I20260812 03:13:45.069228 31003 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:32837
I20260812 03:13:45.069314 31226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:32837 every 8 connection(s)
I20260812 03:13:45.070554 31003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:13:45.072775 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 31003
I20260812 03:13:45.072955 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:13:45.073246 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.058185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:46215 (local address 127.16.217.126:39749)
0812 03:13:45.058368 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:45.058374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:45.059277 (+ 903us) server_negotiation.cc:407] Connection header received
0812 03:13:45.060283 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:45.060287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:45.060354 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:45.060416 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:45.062252 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.063041 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.064501 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.064863 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.065022 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.068513 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:45.068516 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:45.068518 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:45.068545 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:45.070391 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:45.071147 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:45.071151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:45.071153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:45.071279 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:45.071851 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:45.071855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:45.071856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:45.072215 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:45.072337 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:45.072890 (+ 553us) server_negotiation.cc:299] Negotiation successful
0812 03:13:45.073050 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 03:13:45.073278 17253 external_mini_cluster.cc:1428] Setting key 38fd06720f4707a5c2b99f7284f7edd2
I20260812 03:13:45.074144 31055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.058643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39749 (local address 127.16.217.65:46215)
0812 03:13:45.059102 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:45.059154 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:45.060021 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:45.060902 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:45.060912 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:45.061418 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:45.062063 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.062085 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.063171 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.063174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.064370 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.064381 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.065198 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.065211 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.065450 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.066169 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:45.066187 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:45.068180 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:45.070517 (+ 2337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:45.070526 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:45.070542 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:45.070920 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:45.071392 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:45.071394 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:45.071395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:45.071536 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:45.072366 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:45.072371 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:45.072609 (+ 238us) client_negotiation.cc:769] Sending connection context
0812 03:13:45.072839 (+ 230us) client_negotiation.cc:240] Negotiation successful
0812 03:13:45.073188 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":140,"threads_started":1}
I20260812 03:13:45.075687 31227 heartbeater.cc:347] Connected to a master server at 127.16.217.126:39749
I20260812 03:13:45.075898 31227 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:45.076494 31227 heartbeater.cc:511] Master 127.16.217.126:39749 requested a full tablet report, sending...
I20260812 03:13:45.078148 30877 ts_manager.cc:194] Registered new tserver with Master: 652b61c0fc3f41cf8294353cfb3c4613 (127.16.217.65:32837)
I20260812 03:13:45.080082 30877 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:46215
WARNING: no policy specified for oryx/127.16.217.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.217.66@KRBTEST.COM" created.
I20260812 03:13:45.088595 31242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.077892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39749 (local address 127.16.217.65:45173)
0812 03:13:45.078224 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:45.078354 (+ 130us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:45.078466 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:45.081220 (+ 2754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:45.081223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:45.081456 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:45.081589 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.081597 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.082820 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.082823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.083365 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.083372 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.083978 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.083981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.084049 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.084640 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:45.084652 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:45.084984 (+ 332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:45.086829 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:45.086832 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:45.086834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:45.087067 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:45.087493 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:45.087496 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:45.087497 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:45.087584 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:45.087997 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:45.088000 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:45.088123 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 03:13:45.088221 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 03:13:45.088414 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":110,"threads_started":1}
I20260812 03:13:45.088677 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.080836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:45173 (local address 127.16.217.126:39749)
0812 03:13:45.080973 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:45.080977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:45.080992 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:13:45.081034 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:45.081037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:45.081099 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:45.081191 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:45.081733 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.082671 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.083486 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.083862 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.083989 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.085145 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:45.085149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:45.085152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:45.085188 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:45.086720 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:45.087194 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:45.087197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:45.087198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:45.087356 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:45.087710 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:45.087713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:45.087714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:45.087902 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:45.088033 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:45.088319 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 03:13:45.088485 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.66.keytab.
Entry for principal oryx/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.66.keytab.
I20260812 03:13:45.105183 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39749
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.66.keytab
--principal=oryx/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:45.228770 31245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:45.229048 31245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:45.229233 31245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:45.233495 31245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:45.233552 31245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:45.233594 31245 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:13:45.233628 31245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:13:45.238950 31245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.66.keytab
--principal=oryx/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.217.126:39749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:45.240067 31245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:45.241075 31245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:45.251103 31255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:45.251163 31252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:45.251413 31245 server_base.cc:1034] running on GCE node
W20260812 03:13:45.251410 31253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:45.251891 31245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:45.252626 31245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:45.253815 31245 hybrid_clock.cc:648] HybridClock initialized: now 1786504425253802 us; error 31 us; skew 500 ppm
Aug 12 03:13:45 dist-test-slave-7nm7 krb5kdc[30804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504425, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:45.256685 31245 init.cc:377] Logged in from keytab as oryx/127.16.217.66@KRBTEST.COM (short username oryx)
I20260812 03:13:45.257536 31245 webserver.cc:460] Webserver started at http://127.16.217.66:39987/ using document root <none> and password file <none>
I20260812 03:13:45.258077 31245 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:45.258172 31245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:45.258446 31245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:45.260572 31245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "61b184da45bd41728fa4bd011355324d"
format_stamp: "Formatted at 2026-08-12 03:13:45 on dist-test-slave-7nm7"
server_key: "b60e6aa97d9b78fbbe02c5f715f8e832"
server_key_iv: "d97a8c8fec732706a329eafa1b102c79"
server_key_version: "encryptionkey@0"
I20260812 03:13:45.261166 31245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "61b184da45bd41728fa4bd011355324d"
format_stamp: "Formatted at 2026-08-12 03:13:45 on dist-test-slave-7nm7"
server_key: "b60e6aa97d9b78fbbe02c5f715f8e832"
server_key_iv: "d97a8c8fec732706a329eafa1b102c79"
server_key_version: "encryptionkey@0"
I20260812 03:13:45.265204 31245 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 03:13:45.268149 31267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:45.269505 31245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:45.269645 31245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "61b184da45bd41728fa4bd011355324d"
format_stamp: "Formatted at 2026-08-12 03:13:45 on dist-test-slave-7nm7"
server_key: "b60e6aa97d9b78fbbe02c5f715f8e832"
server_key_iv: "d97a8c8fec732706a329eafa1b102c79"
server_key_version: "encryptionkey@0"
I20260812 03:13:45.269759 31245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:45.289975 31245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:45.293473 31245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:45.293708 31245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:45.295586 31245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:45.295665 31245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:45.295722 31245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:45.295751 31245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:13:45 dist-test-slave-7nm7 krb5kdc[30804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504425, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM
I20260812 03:13:45.307879 31245 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:40321
I20260812 03:13:45.307996 31455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:40321 every 8 connection(s)
I20260812 03:13:45.309101 31245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:13:45.311496 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.296177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:42107 (local address 127.16.217.126:39749)
0812 03:13:45.296310 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:45.296317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:45.297490 (+ 1173us) server_negotiation.cc:407] Connection header received
0812 03:13:45.298829 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:45.298833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:45.298896 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:45.299010 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:45.300625 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.301553 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.303135 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.303552 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.303716 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.307192 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:45.307196 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:45.307198 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:45.307224 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:45.308973 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:45.309711 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:45.309715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:45.309717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:45.309824 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:45.310302 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:45.310306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:45.310308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:45.310523 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:45.310657 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:45.311160 (+ 503us) server_negotiation.cc:299] Negotiation successful
0812 03:13:45.311292 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:13:45.312258 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.296716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39749 (local address 127.16.217.66:42107)
0812 03:13:45.297324 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:45.297381 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:45.298593 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:45.299201 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:45.299208 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:45.299706 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:45.300446 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.300466 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.301784 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.301789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.302923 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.302982 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.303771 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.303775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.304020 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.304680 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:45.304717 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:45.306998 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:45.309140 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:45.309147 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:45.309162 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:45.309561 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:45.309938 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:45.309943 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:45.309945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:45.310111 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:45.310657 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:45.310665 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:45.310897 (+ 232us) client_negotiation.cc:769] Sending connection context
0812 03:13:45.311100 (+ 203us) client_negotiation.cc:240] Negotiation successful
0812 03:13:45.311456 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":145,"threads_started":1}
I20260812 03:13:45.313329 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 31245
I20260812 03:13:45.313450 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:13:45.313644 31457 heartbeater.cc:347] Connected to a master server at 127.16.217.126:39749
I20260812 03:13:45.313796 31457 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:45.313820 17253 external_mini_cluster.cc:1428] Setting key 9c24408357b152d19428efdd3fd2c218
I20260812 03:13:45.314441 31457 heartbeater.cc:511] Master 127.16.217.126:39749 requested a full tablet report, sending...
I20260812 03:13:45.315555 30880 ts_manager.cc:194] Registered new tserver with Master: 61b184da45bd41728fa4bd011355324d (127.16.217.66:40321)
I20260812 03:13:45.316385 30880 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:42107
WARNING: no policy specified for oryx/127.16.217.67@KRBTEST.COM; defaulting to no policy
I20260812 03:13:45.325152 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.317716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:60227 (local address 127.16.217.126:39749)
0812 03:13:45.317864 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:45.317871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:45.317888 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:13:45.317931 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:45.317936 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:45.317987 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:45.318088 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:45.318647 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.319264 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.320089 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.320336 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.320453 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.321718 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:45.321722 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:45.321724 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:45.321749 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:45.323363 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:45.323826 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:45.323830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:45.323832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:45.323917 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:45.324239 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:45.324243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:45.324244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:45.324467 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:45.324604 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:45.324874 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 03:13:45.324987 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:13:45.325165 31466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.315903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39749 (local address 127.16.217.66:60227)
0812 03:13:45.316180 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:45.316201 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:45.316277 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:45.318132 (+ 1855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:45.318135 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:45.318390 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:45.318509 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.318517 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.319389 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.319391 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.319955 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.319964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.320459 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.320463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.320560 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.321101 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:45.321113 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:45.321560 (+ 447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:45.323494 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:45.323501 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:45.323503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:45.323717 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:45.324018 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:45.324022 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:45.324024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:45.324137 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:45.324574 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:45.324577 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:45.324676 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 03:13:45.324802 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 03:13:45.324954 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":113,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.67.keytab.
Entry for principal oryx/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.67.keytab.
I20260812 03:13:45.346043 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39749
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.67.keytab
--principal=oryx/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:45.469933 31469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:45.470254 31469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:45.470302 31469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:45.474721 31469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:45.474786 31469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:45.474828 31469 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:13:45.474910 31469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:13:45.480496 31469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.67.keytab
--principal=oryx/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.217.126:39749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:45.481535 31469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:45.482589 31469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:45.492516 31490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:45.492539 31483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:45.492558 31486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:45.492813 31469 server_base.cc:1034] running on GCE node
I20260812 03:13:45.493269 31469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:45.493976 31469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:45.495182 31469 hybrid_clock.cc:648] HybridClock initialized: now 1786504425495149 us; error 51 us; skew 500 ppm
Aug 12 03:13:45 dist-test-slave-7nm7 krb5kdc[30804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504425, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:45.497970 31469 init.cc:377] Logged in from keytab as oryx/127.16.217.67@KRBTEST.COM (short username oryx)
I20260812 03:13:45.498961 31469 webserver.cc:460] Webserver started at http://127.16.217.67:43621/ using document root <none> and password file <none>
I20260812 03:13:45.499545 31469 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:45.499642 31469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:45.499881 31469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:45.501943 31469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "964b0ac977d14f4d9e8b00f4b017e74e"
format_stamp: "Formatted at 2026-08-12 03:13:45 on dist-test-slave-7nm7"
server_key: "4c2bc89b833a9c54965df8bd864d0c06"
server_key_iv: "632ae2909a90708a585fe344aa34a1b2"
server_key_version: "encryptionkey@0"
I20260812 03:13:45.502549 31469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "964b0ac977d14f4d9e8b00f4b017e74e"
format_stamp: "Formatted at 2026-08-12 03:13:45 on dist-test-slave-7nm7"
server_key: "4c2bc89b833a9c54965df8bd864d0c06"
server_key_iv: "632ae2909a90708a585fe344aa34a1b2"
server_key_version: "encryptionkey@0"
I20260812 03:13:45.506453 31469 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:13:45.509038 31500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:45.510447 31469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:45.510582 31469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "964b0ac977d14f4d9e8b00f4b017e74e"
format_stamp: "Formatted at 2026-08-12 03:13:45 on dist-test-slave-7nm7"
server_key: "4c2bc89b833a9c54965df8bd864d0c06"
server_key_iv: "632ae2909a90708a585fe344aa34a1b2"
server_key_version: "encryptionkey@0"
I20260812 03:13:45.510696 31469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:45.527685 31469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:45.530934 31469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:45.531173 31469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:45.533037 31469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:45.533128 31469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:45.533209 31469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:45.533267 31469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:13:45 dist-test-slave-7nm7 krb5kdc[30804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504425, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM
I20260812 03:13:45.545267 31469 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:34477
I20260812 03:13:45.545343 31722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:34477 every 8 connection(s)
I20260812 03:13:45.546334 31469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:13:45.548549 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.533717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:52509 (local address 127.16.217.126:39749)
0812 03:13:45.533899 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:45.533905 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:45.534978 (+ 1073us) server_negotiation.cc:407] Connection header received
0812 03:13:45.536248 (+ 1270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:45.536251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:45.536311 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:45.536415 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:45.538086 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.538754 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.539906 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.540142 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.540260 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.544331 (+ 4071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:45.544335 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:45.544338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:45.544369 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:45.545993 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:45.546646 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:45.546649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:45.546650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:45.546730 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:45.547451 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:45.547456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:45.547459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:45.547715 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:45.547813 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:45.548312 (+ 499us) server_negotiation.cc:299] Negotiation successful
0812 03:13:45.548426 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 03:13:45.549520 31517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.534184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39749 (local address 127.16.217.67:52509)
0812 03:13:45.534784 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:45.534846 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:45.535983 (+ 1137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:45.536623 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:45.536633 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:45.537166 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:45.537896 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.537921 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.538893 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.538897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.539775 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.539785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.540316 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.540321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.540494 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.541124 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:45.541148 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:45.544143 (+ 2995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:45.546120 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:45.546126 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:45.546140 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:45.546548 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:45.546836 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:45.546840 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:45.546842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:45.547033 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:45.547830 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:45.547835 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:45.548063 (+ 228us) client_negotiation.cc:769] Sending connection context
0812 03:13:45.548238 (+ 175us) client_negotiation.cc:240] Negotiation successful
0812 03:13:45.548488 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":117,"threads_started":1}
I20260812 03:13:45.551040 31723 heartbeater.cc:347] Connected to a master server at 127.16.217.126:39749
I20260812 03:13:45.551214 31723 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:45.551719 31723 heartbeater.cc:511] Master 127.16.217.126:39749 requested a full tablet report, sending...
I20260812 03:13:45.552824 30880 ts_manager.cc:194] Registered new tserver with Master: 964b0ac977d14f4d9e8b00f4b017e74e (127.16.217.67:34477)
I20260812 03:13:45.553534 30880 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:52509
I20260812 03:13:45.554553 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 31469
I20260812 03:13:45.554726 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:13:45.555053 17253 external_mini_cluster.cc:1428] Setting key 6601e2b1a910b67ebc77d297ac67262c
I20260812 03:13:45.557261 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:13:45.561317 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.553250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:42227 (local address 127.16.217.126:39749)
0812 03:13:45.553373 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:45.553377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:45.553504 (+ 127us) server_negotiation.cc:407] Connection header received
0812 03:13:45.553648 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:45.553652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:45.553700 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:45.553774 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:45.554269 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.554891 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.555964 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.556292 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.556403 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.557613 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:45.557616 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:45.557619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:45.557645 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:45.559420 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:45.559896 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:45.559899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:45.559901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:45.560031 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:45.560487 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:45.560490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:45.560491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:45.560672 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:45.560757 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:45.561095 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 03:13:45.561190 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:13:45.561445 31732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.553121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39749 (local address 127.16.217.67:42227)
0812 03:13:45.553408 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:45.553422 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:45.553549 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:45.553805 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:45.553808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:45.554002 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:45.554094 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.554099 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.555018 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.555022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.555844 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.555854 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.556430 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.556434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.556521 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.557044 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:45.557060 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:45.557489 (+ 429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:45.559549 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:45.559552 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:45.559554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:45.559748 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:45.560152 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:45.560158 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:45.560160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:45.560359 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:45.560775 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:45.560781 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:45.560894 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 03:13:45.561025 (+ 131us) client_negotiation.cc:240] Negotiation successful
0812 03:13:45.561222 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":114,"threads_started":1}
Aug 12 03:13:45 dist-test-slave-7nm7 krb5kdc[30804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504425, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:13:45 dist-test-slave-7nm7 krb5kdc[30804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:45 dist-test-slave-7nm7 krb5kdc[30804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:13:45.585155 30971 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:45.578614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53500 (local address 127.16.217.126:39749)
0812 03:13:45.578756 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:45.578760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:45.578773 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:13:45.578825 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:45.578827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:45.578866 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:45.578954 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:45.579649 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.580173 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.581008 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.581246 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.581349 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.584960 (+ 3611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53500: BlockingRecv error: recv got EOF from 127.0.0.1:53500 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 03:13:45 dist-test-slave-7nm7 krb5kdc[30804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504425, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM
I20260812 03:13:45.595495 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.586886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53514 (local address 127.16.217.126:39749)
0812 03:13:45.587036 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:45.587039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:45.587161 (+ 122us) server_negotiation.cc:407] Connection header received
0812 03:13:45.587226 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:45.587229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:45.587281 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:45.587354 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:45.587977 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.588479 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.589394 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.589605 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.589688 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.592088 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:45.592091 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:45.592093 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:45.592117 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:45.593797 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:45.594216 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:45.594218 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:45.594220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:45.594395 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:45.594686 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:45.594688 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:45.594689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:45.594859 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:45.594963 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:45.595184 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 03:13:45.595279 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 03:13:45.598589 30880 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:13:45.601698 30880 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:13:45.617223 31770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.611487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:32837 (local address 127.0.0.1:43058)
0812 03:13:45.612270 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:45.612282 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:45.612387 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:45.613023 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:45.613026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:45.613046 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:45.613171 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.613178 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.614502 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.614506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.615838 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.615850 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.616643 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.616647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.616767 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.616804 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 03:13:45.616917 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 03:13:45.617034 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"mutex_wait_us":41,"spinlock_wait_cycles":6272,"thread_start_us":88,"threads_started":1}
I20260812 03:13:45.617272 31766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.611252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:40321 (local address 127.0.0.1:58402)
0812 03:13:45.611658 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:45.611672 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:45.611806 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:45.612765 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:45.612771 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:45.612794 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:45.612951 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.612958 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.614680 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.614683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.615903 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.615910 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.616943 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.616947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.617025 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.617045 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 03:13:45.617107 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 03:13:45.617162 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":21,"thread_start_us":104,"threads_started":1}
I20260812 03:13:45.617421 31772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.611713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58402 (local address 127.16.217.66:40321)
0812 03:13:45.612218 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:45.612231 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:45.612253 (+ 22us) server_negotiation.cc:407] Connection header received
0812 03:13:45.612330 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:45.612337 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:45.612544 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:45.612745 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:45.613039 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.614298 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.616032 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.616831 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.616947 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.617096 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:45.617151 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 03:13:45.617237 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":117,"threads_started":1}
I20260812 03:13:45.617684 31775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.611911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43058 (local address 127.16.217.65:32837)
0812 03:13:45.612258 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:45.612263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:45.612350 (+ 87us) server_negotiation.cc:407] Connection header received
0812 03:13:45.612508 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:45.612513 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:45.612699 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:45.612884 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:45.613308 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.614414 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.615969 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.616508 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.616649 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.616806 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:45.617429 (+ 623us) server_negotiation.cc:299] Negotiation successful
0812 03:13:45.617543 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":87,"threads_started":1}
I20260812 03:13:45.618690 31764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.610817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:34477 (local address 127.0.0.1:36446)
0812 03:13:45.611606 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:45.611656 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:45.611822 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:45.613047 (+ 1225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:45.613048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:45.613064 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:45.613140 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.613144 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.614339 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.614341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.615881 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.615893 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.618297 (+ 2404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.618301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.618388 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.618411 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:13:45.618488 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:13:45.618552 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":22,"thread_start_us":66,"threads_started":1}
I20260812 03:13:45.619108 31765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.610967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36446 (local address 127.16.217.67:34477)
0812 03:13:45.611499 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:45.611518 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:45.611763 (+ 245us) server_negotiation.cc:407] Connection header received
0812 03:13:45.611999 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:45.612005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:45.612496 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:45.612774 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:45.613310 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.614143 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.616032 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.616831 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.616947 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.617096 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:45.618876 (+ 1780us) server_negotiation.cc:299] Negotiation successful
0812 03:13:45.618977 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":172,"threads_started":1}
I20260812 03:13:45.620468 31350 tablet_service.cc:1467] Processing CreateTablet for tablet 2767d96cc804456dabe40f797974aa97 (DEFAULT_TABLE table=test-table [id=50ec2d50b421412aa5d81f14520b4b60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:45.621588 31350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2767d96cc804456dabe40f797974aa97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:45.621320 31599 tablet_service.cc:1467] Processing CreateTablet for tablet 2767d96cc804456dabe40f797974aa97 (DEFAULT_TABLE table=test-table [id=50ec2d50b421412aa5d81f14520b4b60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:45.622318 31599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2767d96cc804456dabe40f797974aa97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:45.624136 31122 tablet_service.cc:1467] Processing CreateTablet for tablet 2767d96cc804456dabe40f797974aa97 (DEFAULT_TABLE table=test-table [id=50ec2d50b421412aa5d81f14520b4b60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:45.625125 31122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2767d96cc804456dabe40f797974aa97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:45.628762 31778 tablet_bootstrap.cc:492] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d: Bootstrap starting.
I20260812 03:13:45.630173 31784 tablet_bootstrap.cc:492] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e: Bootstrap starting.
I20260812 03:13:45.631707 31778 tablet_bootstrap.cc:654] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:45.632355 31784 tablet_bootstrap.cc:654] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:45.632598 31778 log.cc:824] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:45.633174 31784 log.cc:824] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:45.633862 31785 tablet_bootstrap.cc:492] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613: Bootstrap starting.
I20260812 03:13:45.634660 31778 tablet_bootstrap.cc:492] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d: No bootstrap required, opened a new log
I20260812 03:13:45.634874 31778 ts_tablet_manager.cc:1397] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:13:45.635268 31784 tablet_bootstrap.cc:492] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e: No bootstrap required, opened a new log
I20260812 03:13:45.635435 31784 ts_tablet_manager.cc:1397] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:13:45.636031 31785 tablet_bootstrap.cc:654] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:45.636685 31785 log.cc:824] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:45.638291 31785 tablet_bootstrap.cc:492] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613: No bootstrap required, opened a new log
I20260812 03:13:45.638492 31785 ts_tablet_manager.cc:1397] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:13:45.638919 31778 raft_consensus.cc:348] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } }
I20260812 03:13:45.639209 31778 raft_consensus.cc:374] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:45.639264 31778 raft_consensus.cc:729] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b184da45bd41728fa4bd011355324d, State: Initialized, Role: FOLLOWER
I20260812 03:13:45.639163 31784 raft_consensus.cc:348] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } }
I20260812 03:13:45.639396 31784 raft_consensus.cc:374] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:45.639447 31784 raft_consensus.cc:729] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964b0ac977d14f4d9e8b00f4b017e74e, State: Initialized, Role: FOLLOWER
I20260812 03:13:45.639755 31778 consensus_queue.cc:260] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } }
I20260812 03:13:45.639801 31784 consensus_queue.cc:260] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } }
I20260812 03:13:45.640468 31723 heartbeater.cc:503] Master 127.16.217.126:39749 was elected leader, sending a full tablet report...
I20260812 03:13:45.640780 31784 ts_tablet_manager.cc:1428] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 03:13:45.640780 31778 ts_tablet_manager.cc:1428] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:13:45.641129 31457 heartbeater.cc:503] Master 127.16.217.126:39749 was elected leader, sending a full tablet report...
I20260812 03:13:45.643445 31785 raft_consensus.cc:348] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } }
I20260812 03:13:45.643766 31785 raft_consensus.cc:374] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:45.643851 31785 raft_consensus.cc:729] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652b61c0fc3f41cf8294353cfb3c4613, State: Initialized, Role: FOLLOWER
I20260812 03:13:45.644275 31785 consensus_queue.cc:260] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } }
I20260812 03:13:45.644898 31227 heartbeater.cc:503] Master 127.16.217.126:39749 was elected leader, sending a full tablet report...
I20260812 03:13:45.645249 31785 ts_tablet_manager.cc:1428] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 03:13:45.721669 31793 raft_consensus.cc:482] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:45.721889 31793 raft_consensus.cc:504] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } }
I20260812 03:13:45.723125 31793 leader_election.cc:288] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 964b0ac977d14f4d9e8b00f4b017e74e (127.16.217.67:34477), 61b184da45bd41728fa4bd011355324d (127.16.217.66:40321)
I20260812 03:13:45.726174 31791 raft_consensus.cc:482] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:45.726454 31791 raft_consensus.cc:504] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } }
I20260812 03:13:45.727836 31791 leader_election.cc:288] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 964b0ac977d14f4d9e8b00f4b017e74e (127.16.217.67:34477), 652b61c0fc3f41cf8294353cfb3c4613 (127.16.217.65:32837)
I20260812 03:13:45.728854 31765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.723545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:57055 (local address 127.16.217.67:34477)
0812 03:13:45.723691 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:45.723698 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:45.723989 (+ 291us) server_negotiation.cc:407] Connection header received
0812 03:13:45.724266 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:45.724269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:45.724422 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:45.724521 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:45.725111 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.725934 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.727603 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.728258 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.728380 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.728433 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:45.728626 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 03:13:45.728722 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:13:45.728899 31794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.723418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:34477 (local address 127.16.217.65:57055)
0812 03:13:45.723883 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:45.723900 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:45.724035 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:45.724577 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:45.724581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:45.724629 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:45.724882 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.724891 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.726093 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.726099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.727406 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.727418 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.728378 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.728384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.728496 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.728527 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 03:13:45.728610 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 03:13:45.728706 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":183,"spinlock_wait_cycles":30848,"thread_start_us":146,"threads_started":1}
I20260812 03:13:45.729629 31795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.723418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:40321 (local address 127.16.217.65:39141)
0812 03:13:45.723987 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:45.723999 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:45.724083 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:45.724577 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:45.724581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:45.724629 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:45.724882 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.724891 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.726121 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.726125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.727499 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.727511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.729059 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.729076 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.729170 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.729192 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:13:45.729429 (+ 237us) client_negotiation.cc:240] Negotiation successful
0812 03:13:45.729498 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":214,"spinlock_wait_cycles":9984,"thread_start_us":93,"threads_started":1}
I20260812 03:13:45.729647 31639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2767d96cc804456dabe40f797974aa97" candidate_uuid: "652b61c0fc3f41cf8294353cfb3c4613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" is_pre_election: true
I20260812 03:13:45.729779 31772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.723658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:39141 (local address 127.16.217.66:40321)
0812 03:13:45.723812 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:45.723815 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:45.724203 (+ 388us) server_negotiation.cc:407] Connection header received
0812 03:13:45.724265 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:45.724268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:45.724422 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:45.724521 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:45.725033 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.725947 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.727632 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.728255 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.728380 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.728433 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:45.729344 (+ 911us) server_negotiation.cc:299] Negotiation successful
0812 03:13:45.729627 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":185,"server-negotiator.queue_time_us":77}
I20260812 03:13:45.729947 31639 raft_consensus.cc:2454] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652b61c0fc3f41cf8294353cfb3c4613 in term 0.
I20260812 03:13:45.730285 31387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2767d96cc804456dabe40f797974aa97" candidate_uuid: "652b61c0fc3f41cf8294353cfb3c4613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b184da45bd41728fa4bd011355324d" is_pre_election: true
I20260812 03:13:45.730485 31387 raft_consensus.cc:2454] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652b61c0fc3f41cf8294353cfb3c4613 in term 0.
I20260812 03:13:45.730475 31044 leader_election.cc:302] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652b61c0fc3f41cf8294353cfb3c4613, 964b0ac977d14f4d9e8b00f4b017e74e; no voters:
I20260812 03:13:45.730782 31793 raft_consensus.cc:2781] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:13:45.730875 31793 raft_consensus.cc:482] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:13:45.730932 31793 raft_consensus.cc:3037] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:45.732120 31793 raft_consensus.cc:504] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } }
I20260812 03:13:45.732487 31793 leader_election.cc:288] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [CANDIDATE]: Term 1 election: Requested vote from peers 964b0ac977d14f4d9e8b00f4b017e74e (127.16.217.67:34477), 61b184da45bd41728fa4bd011355324d (127.16.217.66:40321)
I20260812 03:13:45.732852 31387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2767d96cc804456dabe40f797974aa97" candidate_uuid: "652b61c0fc3f41cf8294353cfb3c4613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b184da45bd41728fa4bd011355324d"
I20260812 03:13:45.732959 31387 raft_consensus.cc:3037] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:45.733417 31639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2767d96cc804456dabe40f797974aa97" candidate_uuid: "652b61c0fc3f41cf8294353cfb3c4613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964b0ac977d14f4d9e8b00f4b017e74e"
I20260812 03:13:45.733547 31639 raft_consensus.cc:3037] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:45.734200 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.727979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:34477 (local address 127.16.217.66:50885)
0812 03:13:45.728100 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:45.728115 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:45.728195 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:45.729348 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:45.729351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:45.729372 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:45.729515 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.729525 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.731753 (+ 2228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.731760 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.732876 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.732888 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.733793 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.733799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.733922 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.733947 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:13:45.734001 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 03:13:45.734070 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":12}
I20260812 03:13:45.734257 31387 raft_consensus.cc:2454] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652b61c0fc3f41cf8294353cfb3c4613 in term 1.
I20260812 03:13:45.734771 31799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.728735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:50885 (local address 127.16.217.67:34477)
0812 03:13:45.729095 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:45.729107 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:45.729124 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:13:45.729170 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:45.729175 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:45.729231 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:45.730787 (+ 1556us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:45.730886 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.731620 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.733019 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.733628 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.734432 (+ 804us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.734502 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:45.734547 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 03:13:45.734630 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":103,"threads_started":1}
I20260812 03:13:45.734979 31775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.728733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:37637 (local address 127.16.217.65:32837)
0812 03:13:45.728842 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:45.728846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:45.729587 (+ 741us) server_negotiation.cc:407] Connection header received
0812 03:13:45.730033 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:45.730037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:45.730091 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:45.730142 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:45.731727 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.732179 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.733628 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.734261 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.734357 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.734406 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:45.734796 (+ 390us) server_negotiation.cc:299] Negotiation successful
0812 03:13:45.734870 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:13:45.735157 31043 leader_election.cc:302] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61b184da45bd41728fa4bd011355324d, 652b61c0fc3f41cf8294353cfb3c4613; no voters:
I20260812 03:13:45.735277 31639 raft_consensus.cc:2454] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652b61c0fc3f41cf8294353cfb3c4613 in term 1.
I20260812 03:13:45.735320 31793 raft_consensus.cc:2781] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:45.735374 31798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.728274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:32837 (local address 127.16.217.66:37637)
0812 03:13:45.729493 (+ 1219us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:45.729506 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:45.729648 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:45.730299 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:45.730305 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:45.730322 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:45.731578 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.731590 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.732334 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.732336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.733468 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:45.733478 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.734505 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.734508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:45.734621 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.734646 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:13:45.735139 (+ 493us) client_negotiation.cc:240] Negotiation successful
0812 03:13:45.735221 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1151,"thread_start_us":140,"threads_started":1}
I20260812 03:13:45.735543 31793 raft_consensus.cc:686] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 LEADER]: Becoming Leader. State: Replica: 652b61c0fc3f41cf8294353cfb3c4613, State: Running, Role: LEADER
I20260812 03:13:45.735941 31793 consensus_queue.cc:237] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } }
I20260812 03:13:45.736503 31639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2767d96cc804456dabe40f797974aa97" candidate_uuid: "61b184da45bd41728fa4bd011355324d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" is_pre_election: true
I20260812 03:13:45.736553 31152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2767d96cc804456dabe40f797974aa97" candidate_uuid: "61b184da45bd41728fa4bd011355324d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652b61c0fc3f41cf8294353cfb3c4613" is_pre_election: true
I20260812 03:13:45.736703 31639 raft_consensus.cc:2379] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61b184da45bd41728fa4bd011355324d in current term 1: Already voted for candidate 652b61c0fc3f41cf8294353cfb3c4613 in this term.
I20260812 03:13:45.737353 31273 leader_election.cc:302] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61b184da45bd41728fa4bd011355324d; no voters: 652b61c0fc3f41cf8294353cfb3c4613, 964b0ac977d14f4d9e8b00f4b017e74e
I20260812 03:13:45.737816 31791 raft_consensus.cc:2726] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 03:13:45.739737 30880 catalog_manager.cc:5495] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652b61c0fc3f41cf8294353cfb3c4613 (127.16.217.65). New cstate: current_term: 1 leader_uuid: "652b61c0fc3f41cf8294353cfb3c4613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } health_report { overall_health: UNKNOWN } } }
I20260812 03:13:45.774220 31775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:45.770209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43068 (local address 127.16.217.65:32837)
0812 03:13:45.770380 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:45.770384 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:45.770398 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:13:45.770481 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:45.770483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:45.770538 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:45.770632 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:45.770913 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.771429 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.772386 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:45.772621 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:45.772714 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:45.773011 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:45.773721 (+ 710us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:45.773819 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:45.773972 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 03:13:45.774031 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 03:13:45.781395 31639 raft_consensus.cc:1261] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 FOLLOWER]: Refusing update from remote peer 652b61c0fc3f41cf8294353cfb3c4613: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:45.781450 31387 raft_consensus.cc:1261] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d [term 1 FOLLOWER]: Refusing update from remote peer 652b61c0fc3f41cf8294353cfb3c4613: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:45.782143 31793 consensus_queue.cc:1035] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:45.782466 31801 consensus_queue.cc:1035] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [LEADER]: Connected to new peer: Peer: permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:45.790767 31807 mvcc.cc:205] Tried to move back new op lower bound from 7317522127993208832 to 7317522127817986048. Current Snapshot: MvccSnapshot[applied={T|T < 7317522127993208832}]
I20260812 03:13:45.792357 31808 mvcc.cc:205] Tried to move back new op lower bound from 7317522127993208832 to 7317522127817986048. Current Snapshot: MvccSnapshot[applied={T|T < 7317522127993208832}]
I20260812 03:13:45.801009 30877 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:45.801216 30877 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:45.805035 30877 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479: Sending DeleteTablet for 3 replicas of tablet 2767d96cc804456dabe40f797974aa97
I20260812 03:13:45.806533 31599 tablet_service.cc:1514] Processing DeleteTablet for tablet 2767d96cc804456dabe40f797974aa97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:45 UTC) from {username='oryx', principal='oryx/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:36446
I20260812 03:13:45.806679 31122 tablet_service.cc:1514] Processing DeleteTablet for tablet 2767d96cc804456dabe40f797974aa97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:45 UTC) from {username='oryx', principal='oryx/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:43058
I20260812 03:13:45.807235 31826 tablet_replica.cc:330] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e: stopping tablet replica
I20260812 03:13:45.807513 31350 tablet_service.cc:1514] Processing DeleteTablet for tablet 2767d96cc804456dabe40f797974aa97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:45 UTC) from {username='oryx', principal='oryx/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:58402
I20260812 03:13:45.807960 31826 raft_consensus.cc:2229] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:45.808053 31828 tablet_replica.cc:330] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d: stopping tablet replica
I20260812 03:13:45.808454 31828 raft_consensus.cc:2229] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:45.808851 31828 raft_consensus.cc:2258] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:45.808856 31827 tablet_replica.cc:330] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613: stopping tablet replica
I20260812 03:13:45.809026 30877 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 03:13:45.809821 31827 raft_consensus.cc:2229] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 LEADER]: Raft consensus shutting down.
W20260812 03:13:45.809878 30877 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:13:45.810618 31827 raft_consensus.cc:2258] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:45.810708 31828 ts_tablet_manager.cc:1905] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:45.811431 31826 raft_consensus.cc:2258] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:45.813302 31826 ts_tablet_manager.cc:1905] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:45.815177 31828 ts_tablet_manager.cc:1918] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:13:45.815299 31828 log.cc:1196] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/2767d96cc804456dabe40f797974aa97
I20260812 03:13:45.815395 31827 ts_tablet_manager.cc:1905] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:45.815759 31828 ts_tablet_manager.cc:1939] T 2767d96cc804456dabe40f797974aa97 P 61b184da45bd41728fa4bd011355324d: Deleting consensus metadata
I20260812 03:13:45.816885 31826 ts_tablet_manager.cc:1918] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:13:45.817057 31826 log.cc:1196] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/2767d96cc804456dabe40f797974aa97
I20260812 03:13:45.817415 31826 ts_tablet_manager.cc:1939] T 2767d96cc804456dabe40f797974aa97 P 964b0ac977d14f4d9e8b00f4b017e74e: Deleting consensus metadata
I20260812 03:13:45.817466 31122 tablet_service.cc:1467] Processing CreateTablet for tablet 1274b75a8cf24b33b1b99ffc844811b2 (DEFAULT_TABLE table=test-table [id=6f6fb28ef28f474293896d68196afb01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:45.817492 31350 tablet_service.cc:1467] Processing CreateTablet for tablet 1274b75a8cf24b33b1b99ffc844811b2 (DEFAULT_TABLE table=test-table [id=6f6fb28ef28f474293896d68196afb01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:45.817752 31122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1274b75a8cf24b33b1b99ffc844811b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:45.817761 31350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1274b75a8cf24b33b1b99ffc844811b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:45.818645 30858 catalog_manager.cc:4841] TS 61b184da45bd41728fa4bd011355324d (127.16.217.66:40321): tablet 2767d96cc804456dabe40f797974aa97 (table test-table [id=50ec2d50b421412aa5d81f14520b4b60]) successfully deleted
I20260812 03:13:45.819813 30860 catalog_manager.cc:4841] TS 964b0ac977d14f4d9e8b00f4b017e74e (127.16.217.67:34477): tablet 2767d96cc804456dabe40f797974aa97 (table test-table [id=50ec2d50b421412aa5d81f14520b4b60]) successfully deleted
I20260812 03:13:45.820366 31827 ts_tablet_manager.cc:1918] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:13:45.820335 31599 tablet_service.cc:1467] Processing CreateTablet for tablet 1274b75a8cf24b33b1b99ffc844811b2 (DEFAULT_TABLE table=test-table [id=6f6fb28ef28f474293896d68196afb01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:45.820464 31827 log.cc:1196] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/wal/wals/2767d96cc804456dabe40f797974aa97
I20260812 03:13:45.820545 31785 tablet_bootstrap.cc:492] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613: Bootstrap starting.
I20260812 03:13:45.820601 31599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1274b75a8cf24b33b1b99ffc844811b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:45.820793 31827 ts_tablet_manager.cc:1939] T 2767d96cc804456dabe40f797974aa97 P 652b61c0fc3f41cf8294353cfb3c4613: Deleting consensus metadata
I20260812 03:13:45.821580 31785 tablet_bootstrap.cc:654] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:45.822719 31778 tablet_bootstrap.cc:492] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d: Bootstrap starting.
I20260812 03:13:45.822860 31784 tablet_bootstrap.cc:492] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e: Bootstrap starting.
I20260812 03:13:45.823937 31778 tablet_bootstrap.cc:654] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:45.824005 31784 tablet_bootstrap.cc:654] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:45.824358 31785 tablet_bootstrap.cc:492] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613: No bootstrap required, opened a new log
I20260812 03:13:45.824440 31785 ts_tablet_manager.cc:1397] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 03:13:45.824978 31785 raft_consensus.cc:348] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:45.825093 31785 raft_consensus.cc:374] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:45.825152 31785 raft_consensus.cc:729] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652b61c0fc3f41cf8294353cfb3c4613, State: Initialized, Role: FOLLOWER
I20260812 03:13:45.825279 31785 consensus_queue.cc:260] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:45.825598 31785 ts_tablet_manager.cc:1428] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:13:45.825644 31784 tablet_bootstrap.cc:492] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e: No bootstrap required, opened a new log
I20260812 03:13:45.825719 31784 ts_tablet_manager.cc:1397] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:45.826072 30860 catalog_manager.cc:4841] TS 652b61c0fc3f41cf8294353cfb3c4613 (127.16.217.65:32837): tablet 2767d96cc804456dabe40f797974aa97 (table test-table [id=50ec2d50b421412aa5d81f14520b4b60]) successfully deleted
I20260812 03:13:45.826133 31784 raft_consensus.cc:348] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:45.826262 31784 raft_consensus.cc:374] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:45.826304 31784 raft_consensus.cc:729] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964b0ac977d14f4d9e8b00f4b017e74e, State: Initialized, Role: FOLLOWER
I20260812 03:13:45.826463 31784 consensus_queue.cc:260] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:45.826728 31784 ts_tablet_manager.cc:1428] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:45.828282 31778 tablet_bootstrap.cc:492] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d: No bootstrap required, opened a new log
I20260812 03:13:45.828405 31778 ts_tablet_manager.cc:1397] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260812 03:13:45.828892 31778 raft_consensus.cc:348] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:45.829011 31778 raft_consensus.cc:374] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:45.829049 31778 raft_consensus.cc:729] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b184da45bd41728fa4bd011355324d, State: Initialized, Role: FOLLOWER
I20260812 03:13:45.829188 31778 consensus_queue.cc:260] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:45.829566 31778 ts_tablet_manager.cc:1428] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:13:46.006286 31793 raft_consensus.cc:482] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:46.006467 31793 raft_consensus.cc:504] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.006964 31793 leader_election.cc:288] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61b184da45bd41728fa4bd011355324d (127.16.217.66:40321), 964b0ac977d14f4d9e8b00f4b017e74e (127.16.217.67:34477)
I20260812 03:13:46.007454 31387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1274b75a8cf24b33b1b99ffc844811b2" candidate_uuid: "652b61c0fc3f41cf8294353cfb3c4613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b184da45bd41728fa4bd011355324d" is_pre_election: true
I20260812 03:13:46.007535 31639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1274b75a8cf24b33b1b99ffc844811b2" candidate_uuid: "652b61c0fc3f41cf8294353cfb3c4613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" is_pre_election: true
I20260812 03:13:46.007616 31387 raft_consensus.cc:2454] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652b61c0fc3f41cf8294353cfb3c4613 in term 0.
I20260812 03:13:46.007758 31639 raft_consensus.cc:2454] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652b61c0fc3f41cf8294353cfb3c4613 in term 0.
I20260812 03:13:46.007970 31043 leader_election.cc:302] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61b184da45bd41728fa4bd011355324d, 652b61c0fc3f41cf8294353cfb3c4613; no voters:
I20260812 03:13:46.008184 31793 raft_consensus.cc:2781] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:13:46.008246 31793 raft_consensus.cc:482] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:13:46.008277 31793 raft_consensus.cc:3037] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:46.009377 31793 raft_consensus.cc:504] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.009775 31793 leader_election.cc:288] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [CANDIDATE]: Term 1 election: Requested vote from peers 61b184da45bd41728fa4bd011355324d (127.16.217.66:40321), 964b0ac977d14f4d9e8b00f4b017e74e (127.16.217.67:34477)
I20260812 03:13:46.010147 31387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1274b75a8cf24b33b1b99ffc844811b2" candidate_uuid: "652b61c0fc3f41cf8294353cfb3c4613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b184da45bd41728fa4bd011355324d"
I20260812 03:13:46.010195 31639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1274b75a8cf24b33b1b99ffc844811b2" candidate_uuid: "652b61c0fc3f41cf8294353cfb3c4613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964b0ac977d14f4d9e8b00f4b017e74e"
I20260812 03:13:46.010303 31387 raft_consensus.cc:3037] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:46.010321 31639 raft_consensus.cc:3037] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:46.011415 31387 raft_consensus.cc:2454] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652b61c0fc3f41cf8294353cfb3c4613 in term 1.
I20260812 03:13:46.011415 31639 raft_consensus.cc:2454] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652b61c0fc3f41cf8294353cfb3c4613 in term 1.
I20260812 03:13:46.011739 31043 leader_election.cc:302] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61b184da45bd41728fa4bd011355324d, 652b61c0fc3f41cf8294353cfb3c4613; no voters:
I20260812 03:13:46.011951 31793 raft_consensus.cc:2781] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:46.012070 31793 raft_consensus.cc:686] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 LEADER]: Becoming Leader. State: Replica: 652b61c0fc3f41cf8294353cfb3c4613, State: Running, Role: LEADER
I20260812 03:13:46.012212 31793 consensus_queue.cc:237] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.014001 30876 catalog_manager.cc:5495] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652b61c0fc3f41cf8294353cfb3c4613 (127.16.217.65). New cstate: current_term: 1 leader_uuid: "652b61c0fc3f41cf8294353cfb3c4613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } health_report { overall_health: UNKNOWN } } }
W20260812 03:13:46.048408 31725 tablet.cc:2369] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:13:46.060572 31459 tablet.cc:2369] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:13:46.072697 31231 tablet.cc:2369] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 03:13:46 dist-test-slave-7nm7 krb5kdc[30804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504424, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.126@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM
I20260812 03:13:46.098090 31845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:46.084084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39749 (local address 127.0.0.1:53530)
0812 03:13:46.084387 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:46.084404 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:46.084562 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:46.084840 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:46.084843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:46.085328 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:46.085483 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:46.085491 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.086360 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.086363 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:46.086947 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:46.086957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.087414 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.087417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:46.087482 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:46.092932 (+ 5450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:46.092946 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:46.094554 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:46.096414 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:46.096423 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:46.096429 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:46.096695 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:46.096969 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:46.096971 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:46.096972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:46.097071 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:46.097554 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:46.097560 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:46.097693 (+ 133us) client_negotiation.cc:769] Sending connection context
0812 03:13:46.097808 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 03:13:46.097942 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":102,"threads_started":1}
I20260812 03:13:46.098191 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:46.084234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53530 (local address 127.16.217.126:39749)
0812 03:13:46.084369 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:46.084374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:46.084488 (+ 114us) server_negotiation.cc:407] Connection header received
0812 03:13:46.084667 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:46.084670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:46.084732 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:46.084822 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:46.085601 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.086188 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.087078 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.087303 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.087386 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:46.094723 (+ 7337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:46.094726 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:46.094728 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:46.094752 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:46.096237 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:46.096798 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:46.096800 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:46.096802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:46.096877 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:46.097194 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:46.097197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:46.097198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:46.097408 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:46.097501 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:46.097927 (+ 426us) server_negotiation.cc:299] Negotiation successful
0812 03:13:46.098035 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:13:46.102491 30876 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:53530:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 03:13:46.103283 30876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:13:46.108701 31122 tablet_service.cc:1467] Processing CreateTablet for tablet b6c854d0b0334dc3a0605d7748d7e0e1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d15132287444468bb577e274954441bd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:13:46.108652 31350 tablet_service.cc:1467] Processing CreateTablet for tablet b6c854d0b0334dc3a0605d7748d7e0e1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d15132287444468bb577e274954441bd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:13:46.109153 31350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c854d0b0334dc3a0605d7748d7e0e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:46.109156 31122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c854d0b0334dc3a0605d7748d7e0e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:46.112033 31785 tablet_bootstrap.cc:492] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613: Bootstrap starting.
I20260812 03:13:46.112308 31599 tablet_service.cc:1467] Processing CreateTablet for tablet b6c854d0b0334dc3a0605d7748d7e0e1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d15132287444468bb577e274954441bd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:13:46.112680 31599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c854d0b0334dc3a0605d7748d7e0e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:46.112978 31785 tablet_bootstrap.cc:654] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:46.114089 31778 tablet_bootstrap.cc:492] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d: Bootstrap starting.
I20260812 03:13:46.114404 31785 tablet_bootstrap.cc:492] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613: No bootstrap required, opened a new log
I20260812 03:13:46.114492 31785 ts_tablet_manager.cc:1397] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:46.114962 31785 raft_consensus.cc:348] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.115092 31785 raft_consensus.cc:374] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:46.115130 31785 raft_consensus.cc:729] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652b61c0fc3f41cf8294353cfb3c4613, State: Initialized, Role: FOLLOWER
I20260812 03:13:46.115267 31785 consensus_queue.cc:260] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.115345 31778 tablet_bootstrap.cc:654] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:46.115563 31793 tablet_replica.cc:439] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } }
I20260812 03:13:46.115703 31793 tablet_replica.cc:442] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:46.115955 31785 ts_tablet_manager.cc:1428] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:46.116833 31784 tablet_bootstrap.cc:492] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e: Bootstrap starting.
I20260812 03:13:46.117075 31778 tablet_bootstrap.cc:492] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d: No bootstrap required, opened a new log
I20260812 03:13:46.117164 31778 ts_tablet_manager.cc:1397] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:46.117702 31778 raft_consensus.cc:348] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.117843 31778 raft_consensus.cc:374] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:46.117894 31778 raft_consensus.cc:729] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b184da45bd41728fa4bd011355324d, State: Initialized, Role: FOLLOWER
I20260812 03:13:46.117939 31784 tablet_bootstrap.cc:654] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:46.118350 31778 consensus_queue.cc:260] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.118664 31791 tablet_replica.cc:439] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } }
I20260812 03:13:46.118803 31791 tablet_replica.cc:442] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:46.119601 31778 ts_tablet_manager.cc:1428] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:13:46.120015 31784 tablet_bootstrap.cc:492] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e: No bootstrap required, opened a new log
I20260812 03:13:46.120103 31784 ts_tablet_manager.cc:1397] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:46.120591 31784 raft_consensus.cc:348] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.120733 31784 raft_consensus.cc:374] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:46.120779 31784 raft_consensus.cc:729] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964b0ac977d14f4d9e8b00f4b017e74e, State: Initialized, Role: FOLLOWER
I20260812 03:13:46.120930 31784 consensus_queue.cc:260] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.121316 31784 ts_tablet_manager.cc:1428] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:13:46.121305 31789 tablet_replica.cc:439] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } }
I20260812 03:13:46.121492 31789 tablet_replica.cc:442] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:46.474943 31793 raft_consensus.cc:482] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:46.475159 31793 raft_consensus.cc:504] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.475656 31793 leader_election.cc:288] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61b184da45bd41728fa4bd011355324d (127.16.217.66:40321), 964b0ac977d14f4d9e8b00f4b017e74e (127.16.217.67:34477)
I20260812 03:13:46.476070 31387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c854d0b0334dc3a0605d7748d7e0e1" candidate_uuid: "652b61c0fc3f41cf8294353cfb3c4613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b184da45bd41728fa4bd011355324d" is_pre_election: true
I20260812 03:13:46.476189 31639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c854d0b0334dc3a0605d7748d7e0e1" candidate_uuid: "652b61c0fc3f41cf8294353cfb3c4613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" is_pre_election: true
I20260812 03:13:46.476282 31387 raft_consensus.cc:2454] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652b61c0fc3f41cf8294353cfb3c4613 in term 0.
I20260812 03:13:46.476354 31639 raft_consensus.cc:2454] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652b61c0fc3f41cf8294353cfb3c4613 in term 0.
I20260812 03:13:46.476609 31043 leader_election.cc:302] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61b184da45bd41728fa4bd011355324d, 652b61c0fc3f41cf8294353cfb3c4613; no voters:
I20260812 03:13:46.476810 31793 raft_consensus.cc:2781] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:13:46.476874 31793 raft_consensus.cc:482] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:13:46.476925 31793 raft_consensus.cc:3037] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:46.477916 31793 raft_consensus.cc:504] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.478299 31793 leader_election.cc:288] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [CANDIDATE]: Term 1 election: Requested vote from peers 61b184da45bd41728fa4bd011355324d (127.16.217.66:40321), 964b0ac977d14f4d9e8b00f4b017e74e (127.16.217.67:34477)
I20260812 03:13:46.478611 31387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c854d0b0334dc3a0605d7748d7e0e1" candidate_uuid: "652b61c0fc3f41cf8294353cfb3c4613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b184da45bd41728fa4bd011355324d"
I20260812 03:13:46.478700 31639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c854d0b0334dc3a0605d7748d7e0e1" candidate_uuid: "652b61c0fc3f41cf8294353cfb3c4613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964b0ac977d14f4d9e8b00f4b017e74e"
I20260812 03:13:46.478737 31387 raft_consensus.cc:3037] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:46.478819 31639 raft_consensus.cc:3037] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:46.479892 31387 raft_consensus.cc:2454] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652b61c0fc3f41cf8294353cfb3c4613 in term 1.
I20260812 03:13:46.479892 31639 raft_consensus.cc:2454] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652b61c0fc3f41cf8294353cfb3c4613 in term 1.
I20260812 03:13:46.480207 31044 leader_election.cc:302] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652b61c0fc3f41cf8294353cfb3c4613, 964b0ac977d14f4d9e8b00f4b017e74e; no voters:
I20260812 03:13:46.480405 31793 raft_consensus.cc:2781] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:46.480500 31793 raft_consensus.cc:686] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 LEADER]: Becoming Leader. State: Replica: 652b61c0fc3f41cf8294353cfb3c4613, State: Running, Role: LEADER
I20260812 03:13:46.480700 31793 consensus_queue.cc:237] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.481417 31801 tablet_replica.cc:439] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613: TxnStatusTablet state changed. Reason: New leader 652b61c0fc3f41cf8294353cfb3c4613. Latest consensus state: current_term: 1 leader_uuid: "652b61c0fc3f41cf8294353cfb3c4613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } }
I20260812 03:13:46.481559 31801 tablet_replica.cc:442] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:13:46.481812 31853 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 03:13:46.482002 31853 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 03:13:46.482590 30876 catalog_manager.cc:5495] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652b61c0fc3f41cf8294353cfb3c4613 (127.16.217.65). New cstate: current_term: 1 leader_uuid: "652b61c0fc3f41cf8294353cfb3c4613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } health_report { overall_health: UNKNOWN } } }
I20260812 03:13:46.502744 31801 consensus_queue.cc:1035] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:46.506283 31801 consensus_queue.cc:1035] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [LEADER]: Connected to new peer: Peer: permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:46.575167 31845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:46.570791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:32837 (local address 127.0.0.1:43072)
0812 03:13:46.570991 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:46.571012 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:46.571140 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:46.571514 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:46.571516 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:46.571524 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:13:46.571635 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:46.571640 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.572861 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.572865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:46.573663 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:46.573678 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.574179 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.574182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:46.574306 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:46.574356 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:13:46.574796 (+ 440us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:13:46.574820 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:13:46.574861 (+ 41us) client_negotiation.cc:240] Negotiation successful
0812 03:13:46.574936 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 03:13:46.575695 31864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:46.570901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43072 (local address 127.16.217.65:32837)
0812 03:13:46.571203 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:46.571213 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:46.571230 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:13:46.571286 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:46.571289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:46.571343 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:46.571442 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:46.571770 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.572704 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.573824 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.574057 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.574175 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:46.574518 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:46.574617 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:46.575408 (+ 791us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:46.575444 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 03:13:46.575505 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":110,"threads_started":1}
I20260812 03:13:46.578465 31639 raft_consensus.cc:1261] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 FOLLOWER]: Refusing update from remote peer 652b61c0fc3f41cf8294353cfb3c4613: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:46.578472 31387 raft_consensus.cc:1261] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d [term 1 FOLLOWER]: Refusing update from remote peer 652b61c0fc3f41cf8294353cfb3c4613: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:46.578962 31801 consensus_queue.cc:1035] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:46.579175 31793 consensus_queue.cc:1035] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613 [LEADER]: Connected to new peer: Peer: permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:46.580695 31789 tablet_replica.cc:439] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e: TxnStatusTablet state changed. Reason: New leader 652b61c0fc3f41cf8294353cfb3c4613. Latest consensus state: current_term: 1 leader_uuid: "652b61c0fc3f41cf8294353cfb3c4613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } }
I20260812 03:13:46.580819 31789 tablet_replica.cc:442] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:46.581940 31791 tablet_replica.cc:439] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d: TxnStatusTablet state changed. Reason: New leader 652b61c0fc3f41cf8294353cfb3c4613. Latest consensus state: current_term: 1 leader_uuid: "652b61c0fc3f41cf8294353cfb3c4613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } }
I20260812 03:13:46.582053 31791 tablet_replica.cc:442] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:46.582685 31791 tablet_replica.cc:439] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "652b61c0fc3f41cf8294353cfb3c4613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } }
I20260812 03:13:46.582803 31791 tablet_replica.cc:442] T b6c854d0b0334dc3a0605d7748d7e0e1 P 61b184da45bd41728fa4bd011355324d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:46.585690 31875 tablet_replica.cc:439] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "652b61c0fc3f41cf8294353cfb3c4613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } }
I20260812 03:13:46.585855 31875 tablet_replica.cc:442] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:13:46.586854 31861 mvcc.cc:205] Tried to move back new op lower bound from 7317522131260809216 to 7317522130866438144. Current Snapshot: MvccSnapshot[applied={T|T < 7317522131260809216}]
I20260812 03:13:46.582188 31793 tablet_replica.cc:439] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "652b61c0fc3f41cf8294353cfb3c4613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } }
I20260812 03:13:46.588203 31793 tablet_replica.cc:442] T b6c854d0b0334dc3a0605d7748d7e0e1 P 652b61c0fc3f41cf8294353cfb3c4613: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:13:46.588943 31789 tablet_replica.cc:439] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "652b61c0fc3f41cf8294353cfb3c4613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } } }
I20260812 03:13:46.589058 31789 tablet_replica.cc:442] T b6c854d0b0334dc3a0605d7748d7e0e1 P 964b0ac977d14f4d9e8b00f4b017e74e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:13:46.607842 31864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:46.604387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:38099 (local address 127.16.217.65:32837)
0812 03:13:46.604539 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:46.604544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:46.604670 (+ 126us) server_negotiation.cc:407] Connection header received
0812 03:13:46.604854 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:46.604857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:46.604900 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:46.604989 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:46.605250 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.605770 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.606734 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.606966 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.607077 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:46.607357 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:46.607440 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:46.607528 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:46.607611 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 03:13:46.607669 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":60}
I20260812 03:13:46.607846 31882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:46.604250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:32837 (local address 127.16.217.65:38099)
0812 03:13:46.604565 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:46.604579 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:46.604744 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:46.605000 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:46.605003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:46.605009 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:13:46.605145 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:46.605152 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.605896 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.605899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:46.606612 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:46.606621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.607076 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.607080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:46.607168 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:46.607221 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:13:46.607531 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:13:46.607539 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 03:13:46.607604 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 03:13:46.607684 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":1,"thread_start_us":112,"threads_started":1}
I20260812 03:13:46.654317 30876 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:46.654469 30876 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:46.656985 30876 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479: Sending DeleteTablet for 3 replicas of tablet 1274b75a8cf24b33b1b99ffc844811b2
I20260812 03:13:46.657631 31121 tablet_service.cc:1514] Processing DeleteTablet for tablet 1274b75a8cf24b33b1b99ffc844811b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:46 UTC) from {username='oryx', principal='oryx/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:43058
I20260812 03:13:46.657688 31350 tablet_service.cc:1514] Processing DeleteTablet for tablet 1274b75a8cf24b33b1b99ffc844811b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:46 UTC) from {username='oryx', principal='oryx/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:58402
I20260812 03:13:46.657742 31599 tablet_service.cc:1514] Processing DeleteTablet for tablet 1274b75a8cf24b33b1b99ffc844811b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:46 UTC) from {username='oryx', principal='oryx/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:36446
I20260812 03:13:46.658041 31887 tablet_replica.cc:330] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e: stopping tablet replica
I20260812 03:13:46.658051 31888 tablet_replica.cc:330] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d: stopping tablet replica
I20260812 03:13:46.658186 31887 raft_consensus.cc:2229] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:46.658203 31888 raft_consensus.cc:2229] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:46.658350 31887 raft_consensus.cc:2258] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:46.658458 31888 raft_consensus.cc:2258] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:46.659180 31887 ts_tablet_manager.cc:1905] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:46.659381 31888 ts_tablet_manager.cc:1905] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:46.662191 31886 tablet_replica.cc:330] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613: stopping tablet replica
I20260812 03:13:46.662367 31886 raft_consensus.cc:2229] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:13:46.662621 31886 raft_consensus.cc:2258] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:46.663038 31888 ts_tablet_manager.cc:1918] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:13:46.663190 31888 log.cc:1196] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/1274b75a8cf24b33b1b99ffc844811b2
I20260812 03:13:46.663321 31886 ts_tablet_manager.cc:1905] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:46.663467 31887 ts_tablet_manager.cc:1918] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:13:46.663609 31887 log.cc:1196] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/1274b75a8cf24b33b1b99ffc844811b2
I20260812 03:13:46.663766 31888 ts_tablet_manager.cc:1939] T 1274b75a8cf24b33b1b99ffc844811b2 P 61b184da45bd41728fa4bd011355324d: Deleting consensus metadata
I20260812 03:13:46.663918 31887 ts_tablet_manager.cc:1939] T 1274b75a8cf24b33b1b99ffc844811b2 P 964b0ac977d14f4d9e8b00f4b017e74e: Deleting consensus metadata
I20260812 03:13:46.664743 30858 catalog_manager.cc:4841] TS 61b184da45bd41728fa4bd011355324d (127.16.217.66:40321): tablet 1274b75a8cf24b33b1b99ffc844811b2 (table test-table [id=6f6fb28ef28f474293896d68196afb01]) successfully deleted
I20260812 03:13:46.664896 30860 catalog_manager.cc:4841] TS 964b0ac977d14f4d9e8b00f4b017e74e (127.16.217.67:34477): tablet 1274b75a8cf24b33b1b99ffc844811b2 (table test-table [id=6f6fb28ef28f474293896d68196afb01]) successfully deleted
I20260812 03:13:46.666347 31886 ts_tablet_manager.cc:1918] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:13:46.666466 31886 log.cc:1196] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/wal/wals/1274b75a8cf24b33b1b99ffc844811b2
I20260812 03:13:46.666755 31886 ts_tablet_manager.cc:1939] T 1274b75a8cf24b33b1b99ffc844811b2 P 652b61c0fc3f41cf8294353cfb3c4613: Deleting consensus metadata
I20260812 03:13:46.667565 30860 catalog_manager.cc:4841] TS 652b61c0fc3f41cf8294353cfb3c4613 (127.16.217.65:32837): tablet 1274b75a8cf24b33b1b99ffc844811b2 (table test-table [id=6f6fb28ef28f474293896d68196afb01]) successfully deleted
Aug 12 03:13:46 dist-test-slave-7nm7 krb5kdc[30804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:13:46 dist-test-slave-7nm7 krb5kdc[30804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:46 dist-test-slave-7nm7 krb5kdc[30804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:13:46.687495 31898 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:46.680754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53540 (local address 127.16.217.126:39749)
0812 03:13:46.681037 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:46.681043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:46.681173 (+ 130us) server_negotiation.cc:407] Connection header received
0812 03:13:46.681384 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:46.681387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:46.681435 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:46.681534 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:46.682335 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.682898 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.683755 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.683991 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.684128 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:46.687342 (+ 3214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53540: BlockingRecv error: recv got EOF from 127.0.0.1:53540 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":96,"threads_started":1}
Aug 12 03:13:46 dist-test-slave-7nm7 krb5kdc[30804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM
I20260812 03:13:46.698163 31898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:46.689216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53546 (local address 127.16.217.126:39749)
0812 03:13:46.689367 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:46.689372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:46.689474 (+ 102us) server_negotiation.cc:407] Connection header received
0812 03:13:46.689653 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:46.689656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:46.689697 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:46.689786 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:46.690587 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.691109 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.691900 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.692126 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.692233 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:46.694794 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:46.694797 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:46.694799 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:46.694824 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:46.696455 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:46.696868 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:46.696871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:46.696872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:46.696949 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:46.697211 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:46.697214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:46.697215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:46.697437 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:46.697559 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:46.697811 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 03:13:46.697918 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 03:13:46.701085 30876 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53546:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:13:46.701516 30876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:13:46.707023 31350 tablet_service.cc:1467] Processing CreateTablet for tablet 7f7e380a90584949844779f840dab322 (DEFAULT_TABLE table=test-table [id=a9eb94dba7124140a5d266ab208300df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:46.707026 31121 tablet_service.cc:1467] Processing CreateTablet for tablet 7f7e380a90584949844779f840dab322 (DEFAULT_TABLE table=test-table [id=a9eb94dba7124140a5d266ab208300df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:46.707103 31599 tablet_service.cc:1467] Processing CreateTablet for tablet 7f7e380a90584949844779f840dab322 (DEFAULT_TABLE table=test-table [id=a9eb94dba7124140a5d266ab208300df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:46.707389 31121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f7e380a90584949844779f840dab322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:46.707389 31350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f7e380a90584949844779f840dab322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:46.707412 31599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f7e380a90584949844779f840dab322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:46.709472 31912 tablet_bootstrap.cc:492] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e: Bootstrap starting.
I20260812 03:13:46.710521 31912 tablet_bootstrap.cc:654] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:46.710623 31913 tablet_bootstrap.cc:492] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d: Bootstrap starting.
I20260812 03:13:46.710666 31914 tablet_bootstrap.cc:492] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613: Bootstrap starting.
I20260812 03:13:46.711522 31914 tablet_bootstrap.cc:654] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:46.711576 31913 tablet_bootstrap.cc:654] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:46.712432 31912 tablet_bootstrap.cc:492] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e: No bootstrap required, opened a new log
I20260812 03:13:46.712585 31912 ts_tablet_manager.cc:1397] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:46.712997 31913 tablet_bootstrap.cc:492] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d: No bootstrap required, opened a new log
I20260812 03:13:46.713099 31913 ts_tablet_manager.cc:1397] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:46.713380 31914 tablet_bootstrap.cc:492] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613: No bootstrap required, opened a new log
I20260812 03:13:46.713406 31912 raft_consensus.cc:348] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.713501 31914 ts_tablet_manager.cc:1397] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 03:13:46.713533 31912 raft_consensus.cc:374] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:46.713577 31912 raft_consensus.cc:729] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964b0ac977d14f4d9e8b00f4b017e74e, State: Initialized, Role: FOLLOWER
I20260812 03:13:46.713514 31913 raft_consensus.cc:348] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.713640 31913 raft_consensus.cc:374] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:46.713683 31913 raft_consensus.cc:729] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b184da45bd41728fa4bd011355324d, State: Initialized, Role: FOLLOWER
I20260812 03:13:46.713747 31912 consensus_queue.cc:260] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.713811 31913 consensus_queue.cc:260] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.714006 31914 raft_consensus.cc:348] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.714157 31914 raft_consensus.cc:374] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:46.714172 31913 ts_tablet_manager.cc:1428] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:46.714205 31914 raft_consensus.cc:729] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652b61c0fc3f41cf8294353cfb3c4613, State: Initialized, Role: FOLLOWER
I20260812 03:13:46.714372 31914 consensus_queue.cc:260] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.714639 31912 ts_tablet_manager.cc:1428] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:13:46.714690 31914 ts_tablet_manager.cc:1428] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 03:13:46.768003 31793 raft_consensus.cc:482] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:46.768181 31793 raft_consensus.cc:504] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.768713 31793 leader_election.cc:288] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61b184da45bd41728fa4bd011355324d (127.16.217.66:40321), 964b0ac977d14f4d9e8b00f4b017e74e (127.16.217.67:34477)
I20260812 03:13:46.769341 31387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f7e380a90584949844779f840dab322" candidate_uuid: "652b61c0fc3f41cf8294353cfb3c4613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b184da45bd41728fa4bd011355324d" is_pre_election: true
I20260812 03:13:46.769341 31639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f7e380a90584949844779f840dab322" candidate_uuid: "652b61c0fc3f41cf8294353cfb3c4613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" is_pre_election: true
I20260812 03:13:46.769500 31387 raft_consensus.cc:2454] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652b61c0fc3f41cf8294353cfb3c4613 in term 0.
I20260812 03:13:46.769500 31639 raft_consensus.cc:2454] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652b61c0fc3f41cf8294353cfb3c4613 in term 0.
I20260812 03:13:46.769884 31043 leader_election.cc:302] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61b184da45bd41728fa4bd011355324d, 652b61c0fc3f41cf8294353cfb3c4613; no voters:
I20260812 03:13:46.770114 31793 raft_consensus.cc:2781] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:13:46.770190 31793 raft_consensus.cc:482] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:13:46.770219 31793 raft_consensus.cc:3037] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:46.771318 31793 raft_consensus.cc:504] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.771725 31793 leader_election.cc:288] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [CANDIDATE]: Term 1 election: Requested vote from peers 61b184da45bd41728fa4bd011355324d (127.16.217.66:40321), 964b0ac977d14f4d9e8b00f4b017e74e (127.16.217.67:34477)
I20260812 03:13:46.772176 31639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f7e380a90584949844779f840dab322" candidate_uuid: "652b61c0fc3f41cf8294353cfb3c4613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964b0ac977d14f4d9e8b00f4b017e74e"
I20260812 03:13:46.772312 31639 raft_consensus.cc:3037] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:46.772482 31387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f7e380a90584949844779f840dab322" candidate_uuid: "652b61c0fc3f41cf8294353cfb3c4613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b184da45bd41728fa4bd011355324d"
I20260812 03:13:46.772723 31387 raft_consensus.cc:3037] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:46.773494 31639 raft_consensus.cc:2454] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652b61c0fc3f41cf8294353cfb3c4613 in term 1.
I20260812 03:13:46.773890 31044 leader_election.cc:302] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652b61c0fc3f41cf8294353cfb3c4613, 964b0ac977d14f4d9e8b00f4b017e74e; no voters:
I20260812 03:13:46.774147 31793 raft_consensus.cc:2781] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:46.774302 31793 raft_consensus.cc:686] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 LEADER]: Becoming Leader. State: Replica: 652b61c0fc3f41cf8294353cfb3c4613, State: Running, Role: LEADER
I20260812 03:13:46.774292 31387 raft_consensus.cc:2454] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652b61c0fc3f41cf8294353cfb3c4613 in term 1.
I20260812 03:13:46.774458 31793 consensus_queue.cc:237] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.776775 30876 catalog_manager.cc:5495] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652b61c0fc3f41cf8294353cfb3c4613 (127.16.217.65). New cstate: current_term: 1 leader_uuid: "652b61c0fc3f41cf8294353cfb3c4613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } health_report { overall_health: UNKNOWN } } }
I20260812 03:13:46.798957 31864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:46.795745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43078 (local address 127.16.217.65:32837)
0812 03:13:46.795904 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:46.795911 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:46.795941 (+ 30us) server_negotiation.cc:407] Connection header received
0812 03:13:46.796017 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:46.796019 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:46.796065 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:46.796155 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:46.796418 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.796941 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.797805 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.798037 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.798148 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:46.798455 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:46.798534 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:46.798620 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:46.798748 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 03:13:46.798805 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:13:46.801430 31639 raft_consensus.cc:1261] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 FOLLOWER]: Refusing update from remote peer 652b61c0fc3f41cf8294353cfb3c4613: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:46.801561 31387 raft_consensus.cc:1261] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d [term 1 FOLLOWER]: Refusing update from remote peer 652b61c0fc3f41cf8294353cfb3c4613: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:46.802168 31793 consensus_queue.cc:1035] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [LEADER]: Connected to new peer: Peer: permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:46.802385 31801 consensus_queue.cc:1035] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:46.809037 30876 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:46.809265 30876 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:46.812148 30876 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479: Sending DeleteTablet for 3 replicas of tablet 7f7e380a90584949844779f840dab322
I20260812 03:13:46.812860 31121 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f7e380a90584949844779f840dab322 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:46 UTC) from {username='oryx', principal='oryx/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:43058
I20260812 03:13:46.812860 31599 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f7e380a90584949844779f840dab322 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:46 UTC) from {username='oryx', principal='oryx/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:36446
I20260812 03:13:46.812932 31350 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f7e380a90584949844779f840dab322 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:46 UTC) from {username='oryx', principal='oryx/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:58402
I20260812 03:13:46.813241 31887 tablet_replica.cc:330] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e: stopping tablet replica
I20260812 03:13:46.813247 31888 tablet_replica.cc:330] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d: stopping tablet replica
I20260812 03:13:46.813372 31888 raft_consensus.cc:2229] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:46.813418 31887 raft_consensus.cc:2229] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:46.813532 31888 raft_consensus.cc:2258] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:46.813642 31887 raft_consensus.cc:2258] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:46.813853 31886 tablet_replica.cc:330] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613: stopping tablet replica
I20260812 03:13:46.813982 31886 raft_consensus.cc:2229] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:13:46.814333 31886 raft_consensus.cc:2258] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:46.814225 30876 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53546:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 03:13:46.814500 31888 ts_tablet_manager.cc:1905] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 03:13:46.814693 30876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:13:46.815229 31886 ts_tablet_manager.cc:1905] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:46.815719 31887 ts_tablet_manager.cc:1905] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:46.817149 31888 ts_tablet_manager.cc:1918] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:13:46.817220 31888 log.cc:1196] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/7f7e380a90584949844779f840dab322
I20260812 03:13:46.817847 31886 ts_tablet_manager.cc:1918] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:13:46.817920 31886 log.cc:1196] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-0/wal/wals/7f7e380a90584949844779f840dab322
I20260812 03:13:46.818307 31886 ts_tablet_manager.cc:1939] T 7f7e380a90584949844779f840dab322 P 652b61c0fc3f41cf8294353cfb3c4613: Deleting consensus metadata
I20260812 03:13:46.818722 31887 ts_tablet_manager.cc:1918] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:13:46.818823 31887 log.cc:1196] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/7f7e380a90584949844779f840dab322
I20260812 03:13:46.819116 31887 ts_tablet_manager.cc:1939] T 7f7e380a90584949844779f840dab322 P 964b0ac977d14f4d9e8b00f4b017e74e: Deleting consensus metadata
I20260812 03:13:46.819162 30860 catalog_manager.cc:4841] TS 652b61c0fc3f41cf8294353cfb3c4613 (127.16.217.65:32837): tablet 7f7e380a90584949844779f840dab322 (table test-table [id=a9eb94dba7124140a5d266ab208300df]) successfully deleted
I20260812 03:13:46.819257 31888 ts_tablet_manager.cc:1939] T 7f7e380a90584949844779f840dab322 P 61b184da45bd41728fa4bd011355324d: Deleting consensus metadata
I20260812 03:13:46.819841 30860 catalog_manager.cc:4841] TS 964b0ac977d14f4d9e8b00f4b017e74e (127.16.217.67:34477): tablet 7f7e380a90584949844779f840dab322 (table test-table [id=a9eb94dba7124140a5d266ab208300df]) successfully deleted
I20260812 03:13:46.820444 31121 tablet_service.cc:1467] Processing CreateTablet for tablet 67a6f41623f94217aabf134aa47d8469 (DEFAULT_TABLE table=test-table [id=31abb1c239e54f10ad2986e4b3bb505f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:46.820619 31599 tablet_service.cc:1467] Processing CreateTablet for tablet 67a6f41623f94217aabf134aa47d8469 (DEFAULT_TABLE table=test-table [id=31abb1c239e54f10ad2986e4b3bb505f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:46.820726 31121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67a6f41623f94217aabf134aa47d8469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:46.820753 30858 catalog_manager.cc:4841] TS 61b184da45bd41728fa4bd011355324d (127.16.217.66:40321): tablet 7f7e380a90584949844779f840dab322 (table test-table [id=a9eb94dba7124140a5d266ab208300df]) successfully deleted
I20260812 03:13:46.820883 31599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67a6f41623f94217aabf134aa47d8469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:46.822707 31914 tablet_bootstrap.cc:492] T 67a6f41623f94217aabf134aa47d8469 P 652b61c0fc3f41cf8294353cfb3c4613: Bootstrap starting.
I20260812 03:13:46.822737 31912 tablet_bootstrap.cc:492] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e: Bootstrap starting.
I20260812 03:13:46.822948 31350 tablet_service.cc:1467] Processing CreateTablet for tablet 67a6f41623f94217aabf134aa47d8469 (DEFAULT_TABLE table=test-table [id=31abb1c239e54f10ad2986e4b3bb505f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:13:46.823167 31350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67a6f41623f94217aabf134aa47d8469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:46.823592 31914 tablet_bootstrap.cc:654] T 67a6f41623f94217aabf134aa47d8469 P 652b61c0fc3f41cf8294353cfb3c4613: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:46.823625 31912 tablet_bootstrap.cc:654] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:46.825047 31914 tablet_bootstrap.cc:492] T 67a6f41623f94217aabf134aa47d8469 P 652b61c0fc3f41cf8294353cfb3c4613: No bootstrap required, opened a new log
I20260812 03:13:46.825151 31914 ts_tablet_manager.cc:1397] T 67a6f41623f94217aabf134aa47d8469 P 652b61c0fc3f41cf8294353cfb3c4613: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:13:46.825206 31913 tablet_bootstrap.cc:492] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d: Bootstrap starting.
I20260812 03:13:46.825345 31912 tablet_bootstrap.cc:492] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e: No bootstrap required, opened a new log
I20260812 03:13:46.825420 31912 ts_tablet_manager.cc:1397] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:46.825575 31914 raft_consensus.cc:348] T 67a6f41623f94217aabf134aa47d8469 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.825686 31914 raft_consensus.cc:374] T 67a6f41623f94217aabf134aa47d8469 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:46.825718 31914 raft_consensus.cc:729] T 67a6f41623f94217aabf134aa47d8469 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652b61c0fc3f41cf8294353cfb3c4613, State: Initialized, Role: FOLLOWER
I20260812 03:13:46.825845 31914 consensus_queue.cc:260] T 67a6f41623f94217aabf134aa47d8469 P 652b61c0fc3f41cf8294353cfb3c4613 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.825889 31912 raft_consensus.cc:348] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.825987 31912 raft_consensus.cc:374] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:46.826020 31912 raft_consensus.cc:729] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964b0ac977d14f4d9e8b00f4b017e74e, State: Initialized, Role: FOLLOWER
I20260812 03:13:46.826119 31913 tablet_bootstrap.cc:654] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:46.826145 31914 ts_tablet_manager.cc:1428] T 67a6f41623f94217aabf134aa47d8469 P 652b61c0fc3f41cf8294353cfb3c4613: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:13:46.826138 31912 consensus_queue.cc:260] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.826411 31912 ts_tablet_manager.cc:1428] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:13:46.827800 31913 tablet_bootstrap.cc:492] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d: No bootstrap required, opened a new log
I20260812 03:13:46.827891 31913 ts_tablet_manager.cc:1397] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:46.828399 31913 raft_consensus.cc:348] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.828533 31913 raft_consensus.cc:374] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:46.828572 31913 raft_consensus.cc:729] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b184da45bd41728fa4bd011355324d, State: Initialized, Role: FOLLOWER
I20260812 03:13:46.828694 31913 consensus_queue.cc:260] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.829483 31913 ts_tablet_manager.cc:1428] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:13:46.948240 31789 raft_consensus.cc:482] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:13:46.948408 31789 raft_consensus.cc:504] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.949556 31789 leader_election.cc:288] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652b61c0fc3f41cf8294353cfb3c4613 (127.16.217.65:32837), 61b184da45bd41728fa4bd011355324d (127.16.217.66:40321)
I20260812 03:13:46.955155 31864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:46.949889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:47711 (local address 127.16.217.65:32837)
0812 03:13:46.950023 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:46.950028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:46.950338 (+ 310us) server_negotiation.cc:407] Connection header received
0812 03:13:46.950488 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:46.950492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:46.950548 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:46.950646 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:46.951024 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.951824 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.953766 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.954433 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.954589 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:46.954645 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:46.954949 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 03:13:46.955022 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:13:46.955191 31941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:46.949960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:39293 (local address 127.16.217.66:40321)
0812 03:13:46.950214 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:46.950217 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:46.950385 (+ 168us) server_negotiation.cc:407] Connection header received
0812 03:13:46.950522 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:46.950525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:46.950602 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:46.950688 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:13:46.951024 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.951824 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.953766 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.954479 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.954589 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:46.954645 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:46.954948 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 03:13:46.955021 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":103,"threads_started":1}
I20260812 03:13:46.955397 31939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:46.949842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:40321 (local address 127.16.217.67:39293)
0812 03:13:46.950315 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:46.950328 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:46.950417 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:46.950699 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:46.950701 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:46.950712 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:46.950903 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:46.950910 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.951994 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.951998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:46.953616 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:46.953628 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.954590 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.954593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:46.954784 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:46.954832 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 03:13:46.954913 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 03:13:46.955033 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":246,"spinlock_wait_cycles":2816,"thread_start_us":127,"threads_started":1}
I20260812 03:13:46.955596 31937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:46.949794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:32837 (local address 127.16.217.67:47711)
0812 03:13:46.950209 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:46.950227 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:46.950370 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:46.950683 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:46.950688 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:46.950708 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:13:46.950903 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:46.950910 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.951994 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.951998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:46.953616 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:46.953628 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.954589 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.954593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:46.954789 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:46.954834 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 03:13:46.954913 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 03:13:46.955033 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":276,"thread_start_us":92,"threads_started":1}
I20260812 03:13:46.956012 31387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67a6f41623f94217aabf134aa47d8469" candidate_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b184da45bd41728fa4bd011355324d" is_pre_election: true
I20260812 03:13:46.956097 31152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67a6f41623f94217aabf134aa47d8469" candidate_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652b61c0fc3f41cf8294353cfb3c4613" is_pre_election: true
I20260812 03:13:46.956203 31387 raft_consensus.cc:2454] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 964b0ac977d14f4d9e8b00f4b017e74e in term 0.
I20260812 03:13:46.956388 31152 raft_consensus.cc:2454] T 67a6f41623f94217aabf134aa47d8469 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 964b0ac977d14f4d9e8b00f4b017e74e in term 0.
I20260812 03:13:46.956588 31505 leader_election.cc:302] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61b184da45bd41728fa4bd011355324d, 964b0ac977d14f4d9e8b00f4b017e74e; no voters:
I20260812 03:13:46.956902 31789 raft_consensus.cc:2781] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:13:46.956969 31789 raft_consensus.cc:482] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:13:46.957010 31789 raft_consensus.cc:3037] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:46.957995 31789 raft_consensus.cc:504] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.958415 31789 leader_election.cc:288] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [CANDIDATE]: Term 1 election: Requested vote from peers 652b61c0fc3f41cf8294353cfb3c4613 (127.16.217.65:32837), 61b184da45bd41728fa4bd011355324d (127.16.217.66:40321)
I20260812 03:13:46.958765 31387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67a6f41623f94217aabf134aa47d8469" candidate_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b184da45bd41728fa4bd011355324d"
I20260812 03:13:46.958865 31387 raft_consensus.cc:3037] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:46.959755 31387 raft_consensus.cc:2454] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 964b0ac977d14f4d9e8b00f4b017e74e in term 1.
I20260812 03:13:46.960021 31152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67a6f41623f94217aabf134aa47d8469" candidate_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652b61c0fc3f41cf8294353cfb3c4613"
I20260812 03:13:46.960058 31505 leader_election.cc:302] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61b184da45bd41728fa4bd011355324d, 964b0ac977d14f4d9e8b00f4b017e74e; no voters:
I20260812 03:13:46.960180 31152 raft_consensus.cc:3037] T 67a6f41623f94217aabf134aa47d8469 P 652b61c0fc3f41cf8294353cfb3c4613 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:46.960268 31789 raft_consensus.cc:2781] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:46.960489 31789 raft_consensus.cc:686] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 LEADER]: Becoming Leader. State: Replica: 964b0ac977d14f4d9e8b00f4b017e74e, State: Running, Role: LEADER
I20260812 03:13:46.960750 31789 consensus_queue.cc:237] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } }
I20260812 03:13:46.961411 31152 raft_consensus.cc:2454] T 67a6f41623f94217aabf134aa47d8469 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 964b0ac977d14f4d9e8b00f4b017e74e in term 1.
I20260812 03:13:46.963480 30876 catalog_manager.cc:5495] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e reported cstate change: term changed from 0 to 1, leader changed from <none> to 964b0ac977d14f4d9e8b00f4b017e74e (127.16.217.67). New cstate: current_term: 1 leader_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "964b0ac977d14f4d9e8b00f4b017e74e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34477 } health_report { overall_health: HEALTHY } } }
I20260812 03:13:46.991272 31945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:46.987116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36448 (local address 127.16.217.67:34477)
0812 03:13:46.987474 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:46.987482 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:46.987495 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:13:46.987574 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:46.987577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:46.987654 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:46.987842 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:46.988081 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.988659 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.989563 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:46.989823 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:46.989907 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:46.990256 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:46.990843 (+ 587us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:46.990937 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:46.991068 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 03:13:46.991123 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":204,"threads_started":1}
I20260812 03:13:47.004863 31951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:46.999796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:32837 (local address 127.16.217.67:54589)
0812 03:13:47.000168 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:47.000183 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:47.000272 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:47.000828 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:47.000831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:47.000840 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:13:47.000958 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:47.000962 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:47.002382 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:47.002385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:47.003114 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:47.003124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:47.003668 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:47.003672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:47.003848 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:47.003909 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:13:47.004520 (+ 611us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:13:47.004528 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 03:13:47.004606 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:13:47.004686 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":155,"threads_started":1}
I20260812 03:13:47.004982 31864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:47.000066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:54589 (local address 127.16.217.65:32837)
0812 03:13:47.000232 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:47.000236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:47.000251 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:13:47.000609 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:47.000613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:47.000679 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:47.000791 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:47.001240 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:47.002053 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:47.003285 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:47.003515 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:47.003605 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:47.004191 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:47.004343 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:47.004472 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:47.004618 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 03:13:47.004703 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 03:13:47.011842 31387 raft_consensus.cc:1261] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d [term 1 FOLLOWER]: Refusing update from remote peer 964b0ac977d14f4d9e8b00f4b017e74e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:47.011996 31152 raft_consensus.cc:1261] T 67a6f41623f94217aabf134aa47d8469 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 FOLLOWER]: Refusing update from remote peer 964b0ac977d14f4d9e8b00f4b017e74e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:13:47.012470 31789 consensus_queue.cc:1035] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b184da45bd41728fa4bd011355324d" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 40321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:47.013113 31942 consensus_queue.cc:1035] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "652b61c0fc3f41cf8294353cfb3c4613" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 32837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:13:47.032205 31882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:47.028847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:34477 (local address 127.16.217.65:39889)
0812 03:13:47.028997 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:47.029012 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:47.029126 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:47.029443 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:47.029446 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:47.029452 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:13:47.029543 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:47.029547 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:47.030332 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:47.030337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:47.030922 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:47.030931 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:47.031409 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:47.031413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:47.031501 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:47.031529 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:13:47.031811 (+ 282us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:13:47.031821 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 03:13:47.031887 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 03:13:47.031973 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 03:13:47.032186 31945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:47.028942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:39889 (local address 127.16.217.67:34477)
0812 03:13:47.029059 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:47.029071 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:47.029087 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:13:47.029207 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:47.029210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:47.029332 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:47.029414 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:13:47.029669 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:47.030195 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:47.031049 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:47.031290 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:47.031397 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:47.031652 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:13:47.031727 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:13:47.031806 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:47.031896 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 03:13:47.031945 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:13:47.058138 30877 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:47.058367 30877 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:13:47.061058 30877 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86707fb71bee45e6b0a40a55f9543479: Sending DeleteTablet for 3 replicas of tablet 67a6f41623f94217aabf134aa47d8469
I20260812 03:13:47.061933 31599 tablet_service.cc:1514] Processing DeleteTablet for tablet 67a6f41623f94217aabf134aa47d8469 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:47 UTC) from {username='oryx', principal='oryx/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:36446
I20260812 03:13:47.062022 31350 tablet_service.cc:1514] Processing DeleteTablet for tablet 67a6f41623f94217aabf134aa47d8469 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:47 UTC) from {username='oryx', principal='oryx/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:58402
I20260812 03:13:47.062145 31121 tablet_service.cc:1514] Processing DeleteTablet for tablet 67a6f41623f94217aabf134aa47d8469 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:13:47 UTC) from {username='oryx', principal='oryx/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:43058
I20260812 03:13:47.062285 31887 tablet_replica.cc:330] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e: stopping tablet replica
I20260812 03:13:47.062372 31886 tablet_replica.cc:330] T 67a6f41623f94217aabf134aa47d8469 P 652b61c0fc3f41cf8294353cfb3c4613: stopping tablet replica
I20260812 03:13:47.062420 31887 raft_consensus.cc:2229] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:13:47.062490 31886 raft_consensus.cc:2229] T 67a6f41623f94217aabf134aa47d8469 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:47.062606 31886 raft_consensus.cc:2258] T 67a6f41623f94217aabf134aa47d8469 P 652b61c0fc3f41cf8294353cfb3c4613 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:47.062765 31887 raft_consensus.cc:2258] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:47.062964 31888 tablet_replica.cc:330] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d: stopping tablet replica
I20260812 03:13:47.063107 31888 raft_consensus.cc:2229] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:13:47.063235 31888 raft_consensus.cc:2258] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:13:47.063374 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 31003
I20260812 03:13:47.074427 31888 ts_tablet_manager.cc:1905] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:47.074776 31887 ts_tablet_manager.cc:1905] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:13:47.077652 31887 ts_tablet_manager.cc:1918] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:13:47.077724 31887 log.cc:1196] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/67a6f41623f94217aabf134aa47d8469
I20260812 03:13:47.077952 31887 ts_tablet_manager.cc:1939] T 67a6f41623f94217aabf134aa47d8469 P 964b0ac977d14f4d9e8b00f4b017e74e: Deleting consensus metadata
I20260812 03:13:47.078197 31888 ts_tablet_manager.cc:1918] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:13:47.078650 31888 log.cc:1196] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/67a6f41623f94217aabf134aa47d8469
I20260812 03:13:47.079054 31888 ts_tablet_manager.cc:1939] T 67a6f41623f94217aabf134aa47d8469 P 61b184da45bd41728fa4bd011355324d: Deleting consensus metadata
I20260812 03:13:47.079990 30860 catalog_manager.cc:4841] TS 964b0ac977d14f4d9e8b00f4b017e74e (127.16.217.67:34477): tablet 67a6f41623f94217aabf134aa47d8469 (table test-table [id=31abb1c239e54f10ad2986e4b3bb505f]) successfully deleted
I20260812 03:13:47.079990 30858 catalog_manager.cc:4841] TS 61b184da45bd41728fa4bd011355324d (127.16.217.66:40321): tablet 67a6f41623f94217aabf134aa47d8469 (table test-table [id=31abb1c239e54f10ad2986e4b3bb505f]) successfully deleted
W20260812 03:13:47.090687 30860 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.217.65:32837 (error 108)
I20260812 03:13:47.091030 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 31245
I20260812 03:13:47.112551 31965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:47.111749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:32837 (local address 127.0.0.1:43094)
0812 03:13:47.112226 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:47.112379 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":89,"threads_started":1}
W20260812 03:13:47.112736 30860 catalog_manager.cc:4584] TS 652b61c0fc3f41cf8294353cfb3c4613 (127.16.217.65:32837): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 67a6f41623f94217aabf134aa47d8469: Network error: Client connection negotiation failed: client connection to 127.16.217.65:32837: connect: Connection refused (error 111)
I20260812 03:13:47.119567 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 31469
I20260812 03:13:47.139885 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 30836
2026-08-12T03:13:47Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (4111 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:13:47 dist-test-slave-7nm7 krb5kdc[31975](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:13:47 dist-test-slave-7nm7 krb5kdc[31975](info): set up 2 sockets
Aug 12 03:13:47 dist-test-slave-7nm7 krb5kdc[31975](info): commencing operation
krb5kdc: starting...
W20260812 03:13:48.499061 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.322s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:13:48 dist-test-slave-7nm7 krb5kdc[31975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:13:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:13:48.633936 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:35121
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:36431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:35121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:48.754212 32001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:48.754557 32001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:48.754640 32001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:48.758715 32001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:48.758812 32001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:48.758850 32001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:48.758908 32001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:48.758947 32001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:13:48.764891 32001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:36431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:35121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:35121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:48.766127 32001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:48.767252 32001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:48.776085 32008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:48.776091 32012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:48.776091 32007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:48.776599 32001 server_base.cc:1034] running on GCE node
I20260812 03:13:48.777243 32001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:48.778514 32001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:48.779716 32001 hybrid_clock.cc:648] HybridClock initialized: now 1786504428779686 us; error 51 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T03:13:48Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1638 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:13:48 dist-test-slave-7nm7 krb5kdc[32022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:13:48 dist-test-slave-7nm7 krb5kdc[32022](info): set up 2 sockets
Aug 12 03:13:48 dist-test-slave-7nm7 krb5kdc[32022](info): commencing operation
krb5kdc: starting...
W20260812 03:13:50.115816 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:13:50 dist-test-slave-7nm7 krb5kdc[32022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:13:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:13:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:13:50.254799 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:43385
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:39493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:43385
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:50.376583 32050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:50.376878 32050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:50.376921 32050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:50.381026 32050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:50.381104 32050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:50.381125 32050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:50.381142 32050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:50.381158 32050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:13:50.386629 32050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:39493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:43385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:43385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:50.387733 32050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:50.388757 32050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:50.397780 32060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:50.397785 32057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:50.397821 32062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:50.398190 32050 server_base.cc:1034] running on GCE node
I20260812 03:13:50.398840 32050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:50.399991 32050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:50.401196 32050 hybrid_clock.cc:648] HybridClock initialized: now 1786504430401184 us; error 28 us; skew 500 ppm
Aug 12 03:13:50 dist-test-slave-7nm7 krb5kdc[32022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504430, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:50.404141 32050 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:13:50.405043 32050 webserver.cc:460] Webserver started at http://127.16.217.126:45117/ using document root <none> and password file <none>
I20260812 03:13:50.406332 32050 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:50.406420 32050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:50.406618 32050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:50.408680 32050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "aa9d62ef9b654495b5b25de737119656"
format_stamp: "Formatted at 2026-08-12 03:13:50 on dist-test-slave-7nm7"
server_key: "e24d0a380911a87a5f45a289f889bc7b"
server_key_iv: "1a2dd5ee2580d37e0abccbe9fb4ddf86"
server_key_version: "encryptionkey@0"
I20260812 03:13:50.409204 32050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "aa9d62ef9b654495b5b25de737119656"
format_stamp: "Formatted at 2026-08-12 03:13:50 on dist-test-slave-7nm7"
server_key: "e24d0a380911a87a5f45a289f889bc7b"
server_key_iv: "1a2dd5ee2580d37e0abccbe9fb4ddf86"
server_key_version: "encryptionkey@0"
I20260812 03:13:50.413136 32050 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:13:50.415889 32071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:50.417264 32050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:13:50.417408 32050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "aa9d62ef9b654495b5b25de737119656"
format_stamp: "Formatted at 2026-08-12 03:13:50 on dist-test-slave-7nm7"
server_key: "e24d0a380911a87a5f45a289f889bc7b"
server_key_iv: "1a2dd5ee2580d37e0abccbe9fb4ddf86"
server_key_version: "encryptionkey@0"
I20260812 03:13:50.417524 32050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:50.450793 32050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:50.454512 32050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:50.454746 32050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:50.463439 32050 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:43385
I20260812 03:13:50.463521 32161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:43385 every 8 connection(s)
I20260812 03:13:50.464753 32050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:50.468115 32162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:13:50.473754 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 32050
I20260812 03:13:50.473904 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:13:50.474368 17253 external_mini_cluster.cc:1428] Setting key c8672012233b8250756f88a3d2a39651
I20260812 03:13:50.474380 32162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656: Bootstrap starting.
I20260812 03:13:50.477483 32162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656: Neither blocks nor log segments found. Creating new log.
I20260812 03:13:50.478415 32162 log.cc:824] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:50.480581 32162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656: No bootstrap required, opened a new log
Aug 12 03:13:50 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:50.484539 32162 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa9d62ef9b654495b5b25de737119656" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43385 } }
I20260812 03:13:50.484815 32162 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:13:50.484892 32162 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa9d62ef9b654495b5b25de737119656, State: Initialized, Role: FOLLOWER
I20260812 03:13:50.485433 32162 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa9d62ef9b654495b5b25de737119656" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43385 } }
I20260812 03:13:50.485599 32162 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:50.485684 32162 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:50.485802 32162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:13:50.486833 32162 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa9d62ef9b654495b5b25de737119656" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43385 } }
I20260812 03:13:50.487241 32162 leader_election.cc:302] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa9d62ef9b654495b5b25de737119656; no voters:
I20260812 03:13:50.487619 32162 leader_election.cc:288] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:13:50.487766 32169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:13:50.488032 32169 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [term 1 LEADER]: Becoming Leader. State: Replica: aa9d62ef9b654495b5b25de737119656, State: Running, Role: LEADER
I20260812 03:13:50.488397 32169 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa9d62ef9b654495b5b25de737119656" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43385 } }
I20260812 03:13:50.488966 32162 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:50.490149 32171 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa9d62ef9b654495b5b25de737119656. Latest consensus state: current_term: 1 leader_uuid: "aa9d62ef9b654495b5b25de737119656" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa9d62ef9b654495b5b25de737119656" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43385 } } }
I20260812 03:13:50.490295 32171 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:50.490528 32165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:50.476055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53728 (local address 127.16.217.126:43385)
0812 03:13:50.476545 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:50.476568 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:50.476610 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:13:50.477442 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:50.477464 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:50.477968 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:50.478509 (+ 541us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:50.479493 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:50.480516 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:50.481416 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:50.481890 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:50.482074 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:50.484742 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:50.484747 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:50.484763 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:50.484790 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:50.486644 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:50.487124 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:50.487130 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:50.487136 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:50.487283 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:50.487643 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:50.487647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:50.487649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:50.488169 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:50.488371 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:50.489102 (+ 731us) server_negotiation.cc:299] Negotiation successful
0812 03:13:50.489417 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":166,"threads_started":1}
I20260812 03:13:50.491109 32183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:50.491750 32172 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa9d62ef9b654495b5b25de737119656" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa9d62ef9b654495b5b25de737119656" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43385 } } }
I20260812 03:13:50.491897 32172 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:50.494544 32183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:50.501694 32183 catalog_manager.cc:1347] Generated new cluster ID: 54f5169e282640cda7a4fc1a7d822e5c
I20260812 03:13:50.501786 32183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:50.509860 32183 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:13:50.511119 32183 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:50.526453 32183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656: Generated new TSK 0
I20260812 03:13:50.527230 32183 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
I20260812 03:13:50.561527 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:43385
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:39493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:50.680177 32216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:50.680452 32216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:50.680498 32216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:50.684466 32216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:50.684528 32216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:50.684592 32216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:13:50.690326 32216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:39493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:43385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:50.691509 32216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:50.692588 32216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:50.702661 32223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:50.702759 32216 server_base.cc:1034] running on GCE node
W20260812 03:13:50.702651 32221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:50.702677 32225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:50.703336 32216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:50.703972 32216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:50.705170 32216 hybrid_clock.cc:648] HybridClock initialized: now 1786504430705157 us; error 28 us; skew 500 ppm
Aug 12 03:13:50 dist-test-slave-7nm7 krb5kdc[32022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504430, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:50.708355 32216 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:13:50.709339 32216 webserver.cc:460] Webserver started at http://127.16.217.65:35293/ using document root <none> and password file <none>
I20260812 03:13:50.709934 32216 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:50.710029 32216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:50.710342 32216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:50.712379 32216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "a9e79ec439ac4f51b66e7568eca053b7"
format_stamp: "Formatted at 2026-08-12 03:13:50 on dist-test-slave-7nm7"
server_key: "2363407456d596475435cefa23fd9fa8"
server_key_iv: "89271bb02a59e5c9113f7f29abbb5fd5"
server_key_version: "encryptionkey@0"
I20260812 03:13:50.713279 32216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "a9e79ec439ac4f51b66e7568eca053b7"
format_stamp: "Formatted at 2026-08-12 03:13:50 on dist-test-slave-7nm7"
server_key: "2363407456d596475435cefa23fd9fa8"
server_key_iv: "89271bb02a59e5c9113f7f29abbb5fd5"
server_key_version: "encryptionkey@0"
I20260812 03:13:50.717294 32216 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 03:13:50.719990 32239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:50.721163 32216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:13:50.721328 32216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "a9e79ec439ac4f51b66e7568eca053b7"
format_stamp: "Formatted at 2026-08-12 03:13:50 on dist-test-slave-7nm7"
server_key: "2363407456d596475435cefa23fd9fa8"
server_key_iv: "89271bb02a59e5c9113f7f29abbb5fd5"
server_key_version: "encryptionkey@0"
I20260812 03:13:50.721479 32216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:50.744668 32216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:50.747884 32216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:50.748117 32216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:50.748768 32216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:50.749670 32216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:50.749756 32216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:50.749833 32216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:50.749884 32216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:50.759807 32442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:36543 every 8 connection(s)
I20260812 03:13:50.759960 32216 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:36543
I20260812 03:13:50.761049 32216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:13:50.769780 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 32216
I20260812 03:13:50.769951 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:13:50.770588 17253 external_mini_cluster.cc:1428] Setting key 09496a5e7cffbc6d7e1fe4d009d7b582
Aug 12 03:13:50 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504430, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:50.776142 32165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:50.763131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:52065 (local address 127.16.217.126:43385)
0812 03:13:50.763285 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:50.763289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:50.764151 (+ 862us) server_negotiation.cc:407] Connection header received
0812 03:13:50.765681 (+ 1530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:50.765683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:50.765732 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:50.765810 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:50.767386 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:50.767918 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:50.769133 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:50.769374 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:50.769520 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:50.772334 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:50.772340 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:50.772347 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:50.772395 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:50.773901 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:50.774536 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:50.774539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:50.774541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:50.774625 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:50.774994 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:50.774997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:50.774999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:50.775186 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:50.775298 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:50.775822 (+ 524us) server_negotiation.cc:299] Negotiation successful
0812 03:13:50.775953 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:13:50.776996 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:50.763529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:52065)
0812 03:13:50.763976 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:50.764028 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:50.765455 (+ 1427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:50.765981 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:50.765987 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:50.766482 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:50.767201 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:50.767220 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:50.768074 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:50.768077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:50.768983 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:50.769001 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:50.769578 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:50.769581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:50.769734 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:50.770394 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:50.770411 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:50.772156 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:50.774048 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:50.774055 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:50.774071 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:50.774426 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:50.774724 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:50.774728 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:50.774730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:50.774879 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:50.775298 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:50.775306 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:50.775576 (+ 270us) client_negotiation.cc:769] Sending connection context
0812 03:13:50.775766 (+ 190us) client_negotiation.cc:240] Negotiation successful
0812 03:13:50.776110 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":116,"threads_started":1}
I20260812 03:13:50.778427 32443 heartbeater.cc:347] Connected to a master server at 127.16.217.126:43385
I20260812 03:13:50.778709 32443 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:50.779240 32443 heartbeater.cc:511] Master 127.16.217.126:43385 requested a full tablet report, sending...
I20260812 03:13:50.781008 32098 ts_manager.cc:194] Registered new tserver with Master: a9e79ec439ac4f51b66e7568eca053b7 (127.16.217.65:36543)
WARNING: no policy specified for kudu/127.16.217.66@KRBTEST.COM; defaulting to no policy
I20260812 03:13:50.782620 32098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:52065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
I20260812 03:13:50.801440 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:43385
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:39493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:50.920912 32451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:50.921203 32451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:50.921250 32451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:50.925688 32451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:50.925751 32451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:50.925844 32451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:13:50.932662 32451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:39493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:43385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:50.933900 32451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:50.935052 32451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:50.945118 32460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:50.945168 32461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:50.945147 32464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:50.945439 32451 server_base.cc:1034] running on GCE node
I20260812 03:13:50.945823 32451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:50.946554 32451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:50.947718 32451 hybrid_clock.cc:648] HybridClock initialized: now 1786504430947690 us; error 38 us; skew 500 ppm
Aug 12 03:13:50 dist-test-slave-7nm7 krb5kdc[32022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504430, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:50.950765 32451 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:13:50.951807 32451 webserver.cc:460] Webserver started at http://127.16.217.66:42523/ using document root <none> and password file <none>
I20260812 03:13:50.952414 32451 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:50.952483 32451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:50.952716 32451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:50.955185 32451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "789ed83eb5ef4930b5268ba9ae50c10f"
format_stamp: "Formatted at 2026-08-12 03:13:50 on dist-test-slave-7nm7"
server_key: "276fd497288ce7371441f665af036079"
server_key_iv: "6c228393a69161f93124fcd88cb29f9c"
server_key_version: "encryptionkey@0"
I20260812 03:13:50.955824 32451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "789ed83eb5ef4930b5268ba9ae50c10f"
format_stamp: "Formatted at 2026-08-12 03:13:50 on dist-test-slave-7nm7"
server_key: "276fd497288ce7371441f665af036079"
server_key_iv: "6c228393a69161f93124fcd88cb29f9c"
server_key_version: "encryptionkey@0"
I20260812 03:13:50.960457 32451 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:13:50.963465 32480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:50.964846 32451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:13:50.964994 32451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "789ed83eb5ef4930b5268ba9ae50c10f"
format_stamp: "Formatted at 2026-08-12 03:13:50 on dist-test-slave-7nm7"
server_key: "276fd497288ce7371441f665af036079"
server_key_iv: "6c228393a69161f93124fcd88cb29f9c"
server_key_version: "encryptionkey@0"
I20260812 03:13:50.965139 32451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:50.999758 32451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:51.003299 32451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:51.003543 32451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:51.004207 32451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:51.005095 32451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:51.005146 32451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:51.005216 32451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:51.005260 32451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:51.014840 32451 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:34447
I20260812 03:13:51.014863 32670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:34447 every 8 connection(s)
I20260812 03:13:51.015795 32451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:13:51.019320 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 32451
I20260812 03:13:51.019474 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:13:51.019791 17253 external_mini_cluster.cc:1428] Setting key 0d45febd02a6cd1d3e6bdc4f85294a53
Aug 12 03:13:51 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504430, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.217.67@KRBTEST.COM; defaulting to no policy
I20260812 03:13:51.031910 32165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:51.017725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:49307 (local address 127.16.217.126:43385)
0812 03:13:51.017879 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:51.017883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:51.018721 (+ 838us) server_negotiation.cc:407] Connection header received
0812 03:13:51.020458 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:51.020462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:51.020522 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:51.020637 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:51.022097 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.022829 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.024365 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.024661 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.024798 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:51.028083 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:51.028087 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:51.028089 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:51.028124 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:51.029630 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:51.030170 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:51.030174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:51.030176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:51.030314 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:51.030659 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:51.030664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:51.030666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:51.030899 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:51.031031 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:51.031548 (+ 517us) server_negotiation.cc:299] Negotiation successful
0812 03:13:51.031728 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 03:13:51.032976 32676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:51.018074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:49307)
0812 03:13:51.018561 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:51.018607 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:51.020212 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:51.020811 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:51.020818 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:51.021178 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:51.021899 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:51.021925 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.022999 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.023006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:51.024220 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:51.024232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.024840 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.024845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:51.025023 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:51.025837 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:51.025863 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:51.027887 (+ 2024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:51.029766 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:51.029772 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:51.029789 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:51.030062 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:51.030409 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:51.030412 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:51.030414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:51.030561 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:51.031032 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:51.031040 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:51.031307 (+ 267us) client_negotiation.cc:769] Sending connection context
0812 03:13:51.031507 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 03:13:51.031849 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":148,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.67@KRBTEST.COM" created.
I20260812 03:13:51.034420 32672 heartbeater.cc:347] Connected to a master server at 127.16.217.126:43385
I20260812 03:13:51.034708 32672 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:51.035318 32672 heartbeater.cc:511] Master 127.16.217.126:43385 requested a full tablet report, sending...
I20260812 03:13:51.036675 32098 ts_manager.cc:194] Registered new tserver with Master: 789ed83eb5ef4930b5268ba9ae50c10f (127.16.217.66:34447)
I20260812 03:13:51.037478 32098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:49307
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
I20260812 03:13:51.050820 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:43385
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:39493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:51.168572 32684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:51.168851 32684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:51.168896 32684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:51.174536 32684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:51.174610 32684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:51.174672 32684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:13:51.180615 32684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:39493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:43385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:51.181742 32684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:51.182863 32684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:51.192830 32701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:51.192963 32692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:51.192878 32691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:51.193136 32684 server_base.cc:1034] running on GCE node
I20260812 03:13:51.193565 32684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:51.194269 32684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:51.195465 32684 hybrid_clock.cc:648] HybridClock initialized: now 1786504431195440 us; error 50 us; skew 500 ppm
Aug 12 03:13:51 dist-test-slave-7nm7 krb5kdc[32022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504431, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:51.202878 32684 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:13:51.204028 32684 webserver.cc:460] Webserver started at http://127.16.217.67:34771/ using document root <none> and password file <none>
I20260812 03:13:51.204660 32684 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:51.204739 32684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:51.204993 32684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:13:51.207229 32684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "c75b3e08f5d8415a991cabb69c2d51e4"
format_stamp: "Formatted at 2026-08-12 03:13:51 on dist-test-slave-7nm7"
server_key: "e43cfd429611d88873c381f0630d8d9a"
server_key_iv: "335624213a4c2c20f0c1550c6931e37e"
server_key_version: "encryptionkey@0"
I20260812 03:13:51.207828 32684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "c75b3e08f5d8415a991cabb69c2d51e4"
format_stamp: "Formatted at 2026-08-12 03:13:51 on dist-test-slave-7nm7"
server_key: "e43cfd429611d88873c381f0630d8d9a"
server_key_iv: "335624213a4c2c20f0c1550c6931e37e"
server_key_version: "encryptionkey@0"
I20260812 03:13:51.211808 32684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:13:51.214466 32713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:51.215713 32684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:13:51.215854 32684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "c75b3e08f5d8415a991cabb69c2d51e4"
format_stamp: "Formatted at 2026-08-12 03:13:51 on dist-test-slave-7nm7"
server_key: "e43cfd429611d88873c381f0630d8d9a"
server_key_iv: "335624213a4c2c20f0c1550c6931e37e"
server_key_version: "encryptionkey@0"
I20260812 03:13:51.215971 32684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:51.233981 32684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:51.237243 32684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:51.237471 32684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:51.238111 32684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:51.239074 32684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:51.239148 32684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:51.239220 32684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:51.239274 32684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:51.248788 32684 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:39065
I20260812 03:13:51.248888 464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:39065 every 8 connection(s)
I20260812 03:13:51.249789 32684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:13:51.259323 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 32684
Aug 12 03:13:51 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504431, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:13:51.259498 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:13:51.259799 17253 external_mini_cluster.cc:1428] Setting key ce16d768bc3bf2a259e9abda4927a7b0
I20260812 03:13:51.263973 32165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:51.251790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:36547 (local address 127.16.217.126:43385)
0812 03:13:51.251940 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:51.251944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:51.252765 (+ 821us) server_negotiation.cc:407] Connection header received
0812 03:13:51.254162 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:51.254166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:51.254222 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:51.254326 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:51.255484 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.256029 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.257161 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.257400 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.257516 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:51.260098 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:51.260102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:51.260105 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:51.260137 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:51.261758 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:51.262404 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:51.262407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:51.262408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:51.262484 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:51.262843 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:51.262846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:51.262847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:51.263056 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:51.263155 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:51.263743 (+ 588us) server_negotiation.cc:299] Negotiation successful
0812 03:13:51.263851 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:13:51.264772 468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:51.252086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.67:36547)
0812 03:13:51.252599 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:51.252642 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:51.253950 (+ 1308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:51.254489 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:51.254495 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:51.254853 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:51.255327 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:51.255340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.256177 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.256180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:51.257029 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:51.257039 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.257560 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.257565 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:51.257739 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:51.258383 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:51.258400 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:51.259863 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:13:51.261878 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:51.261885 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:51.261906 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:51.262310 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:51.262578 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:13:51.262581 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:13:51.262582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:13:51.262715 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:13:51.263209 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:13:51.263215 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:13:51.263469 (+ 254us) client_negotiation.cc:769] Sending connection context
0812 03:13:51.263653 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 03:13:51.263918 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":121,"threads_started":1}
I20260812 03:13:51.265889 465 heartbeater.cc:347] Connected to a master server at 127.16.217.126:43385
I20260812 03:13:51.266175 465 heartbeater.cc:464] Registering TS with master...
I20260812 03:13:51.266721 465 heartbeater.cc:511] Master 127.16.217.126:43385 requested a full tablet report, sending...
I20260812 03:13:51.267848 32098 ts_manager.cc:194] Registered new tserver with Master: c75b3e08f5d8415a991cabb69c2d51e4 (127.16.217.67:39065)
I20260812 03:13:51.268492 32098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:36547
I20260812 03:13:51.273825 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.65.keytab.
Entry for principal oryx/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.65.keytab.
WARNING: no policy specified for oryx/127.16.217.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.66.keytab.
Entry for principal oryx/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.66.keytab.
WARNING: no policy specified for oryx/127.16.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.67.keytab.
Entry for principal oryx/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.67.keytab.
I20260812 03:13:51.361410 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 32216
I20260812 03:13:51.378655 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 32451
I20260812 03:13:51.389343 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 32684
I20260812 03:13:51.410193 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 32050
I20260812 03:13:51.422204 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:43385
--webserver_interface=127.16.217.126
--webserver_port=45117
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:39493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:43385
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:13:51.544088 487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:51.544399 487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:51.544441 487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:51.548688 487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:13:51.548767 487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:51.548789 487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:51.548805 487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:13:51.548870 487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:13:51.554508 487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:39493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:43385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:43385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=45117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:51.555747 487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:51.556859 487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:51.565495 496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:51.565512 493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:51.565799 487 server_base.cc:1034] running on GCE node
W20260812 03:13:51.565538 501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:51.566632 487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:51.567800 487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:51.568994 487 hybrid_clock.cc:648] HybridClock initialized: now 1786504431568972 us; error 29 us; skew 500 ppm
Aug 12 03:13:51 dist-test-slave-7nm7 krb5kdc[32022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504431, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:51.572350 487 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:13:51.573345 487 webserver.cc:460] Webserver started at http://127.16.217.126:45117/ using document root <none> and password file <none>
I20260812 03:13:51.573947 487 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:51.574024 487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:51.578013 487 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:51.580323 513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:51.581593 487 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:13:51.581755 487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "aa9d62ef9b654495b5b25de737119656"
format_stamp: "Formatted at 2026-08-12 03:13:50 on dist-test-slave-7nm7"
server_key: "e24d0a380911a87a5f45a289f889bc7b"
server_key_iv: "1a2dd5ee2580d37e0abccbe9fb4ddf86"
server_key_version: "encryptionkey@0"
I20260812 03:13:51.582382 487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:51.602684 487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:51.607431 487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:51.607656 487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:51.619359 487 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:43385
I20260812 03:13:51.619379 600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:43385 every 8 connection(s)
I20260812 03:13:51.620474 487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:13:51.625622 603 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:13:51.627557 603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656: Bootstrap starting.
I20260812 03:13:51.630018 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 487
I20260812 03:13:51.630653 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:36543
--local_ip_for_outbound_sockets=127.16.217.65
--tserver_master_addrs=127.16.217.126:43385
--webserver_port=35293
--webserver_interface=127.16.217.65
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:39493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.217.65
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:13:51.646450 603 log.cc:824] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656: Log is configured to *not* fsync() on all Append() calls
I20260812 03:13:51.655078 603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:13:51.655503 603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656: Bootstrap complete.
I20260812 03:13:51.659873 603 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa9d62ef9b654495b5b25de737119656" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43385 } }
I20260812 03:13:51.660256 603 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa9d62ef9b654495b5b25de737119656, State: Initialized, Role: FOLLOWER
I20260812 03:13:51.660794 603 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa9d62ef9b654495b5b25de737119656" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43385 } }
I20260812 03:13:51.660912 603 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:13:51.661010 603 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:13:51.661262 603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:13:51.663350 603 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa9d62ef9b654495b5b25de737119656" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43385 } }
I20260812 03:13:51.663687 603 leader_election.cc:302] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa9d62ef9b654495b5b25de737119656; no voters:
I20260812 03:13:51.664079 603 leader_election.cc:288] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:13:51.664171 611 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:13:51.664350 611 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [term 2 LEADER]: Becoming Leader. State: Replica: aa9d62ef9b654495b5b25de737119656, State: Running, Role: LEADER
I20260812 03:13:51.664646 611 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa9d62ef9b654495b5b25de737119656" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43385 } }
I20260812 03:13:51.665361 603 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:13:51.665607 611 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa9d62ef9b654495b5b25de737119656. Latest consensus state: current_term: 2 leader_uuid: "aa9d62ef9b654495b5b25de737119656" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa9d62ef9b654495b5b25de737119656" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43385 } } }
I20260812 03:13:51.665733 611 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:51.666196 612 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "aa9d62ef9b654495b5b25de737119656" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa9d62ef9b654495b5b25de737119656" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43385 } } }
I20260812 03:13:51.666397 612 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656 [sys.catalog]: This master's current role is: LEADER
I20260812 03:13:51.667846 615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:13:51.671197 615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:13:51.671865 615 catalog_manager.cc:1259] Loaded cluster ID: 54f5169e282640cda7a4fc1a7d822e5c
I20260812 03:13:51.672122 615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:13:51.674567 615 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:13:51.675268 615 catalog_manager.cc:5879] T 00000000000000000000000000000000 P aa9d62ef9b654495b5b25de737119656: Loaded TSK: 0
I20260812 03:13:51.676429 615 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:13:51.752607 609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:51.752890 609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:51.752935 609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:51.756994 609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:51.757063 609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:51.757124 609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:13:51.762832 609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:39493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.65.keytab
--principal=oryx/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:36543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=35293
--tserver_master_addrs=127.16.217.126:43385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:51.763880 609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:51.764967 609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:51.775352 655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:51.775396 653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:51.775396 658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:51.776157 609 server_base.cc:1034] running on GCE node
I20260812 03:13:51.776641 609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:51.777390 609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:51.778612 609 hybrid_clock.cc:648] HybridClock initialized: now 1786504431778586 us; error 45 us; skew 500 ppm
Aug 12 03:13:51 dist-test-slave-7nm7 krb5kdc[32022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504431, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:51.781708 609 init.cc:377] Logged in from keytab as oryx/127.16.217.65@KRBTEST.COM (short username oryx)
I20260812 03:13:51.782866 609 webserver.cc:460] Webserver started at http://127.16.217.65:35293/ using document root <none> and password file <none>
I20260812 03:13:51.783493 609 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:51.783612 609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:51.787881 609 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:13:51.790132 670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:51.791498 609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:13:51.791610 609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "a9e79ec439ac4f51b66e7568eca053b7"
format_stamp: "Formatted at 2026-08-12 03:13:50 on dist-test-slave-7nm7"
server_key: "2363407456d596475435cefa23fd9fa8"
server_key_iv: "89271bb02a59e5c9113f7f29abbb5fd5"
server_key_version: "encryptionkey@0"
I20260812 03:13:51.792189 609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:51.809190 609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:51.812613 609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:51.812872 609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:51.813613 609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:51.814641 609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:51.814697 609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:51.814767 609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:51.814805 609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:51.825388 609 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:36543
I20260812 03:13:51.825421 872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:36543 every 8 connection(s)
I20260812 03:13:51.826474 609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:13:51.829001 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 609
I20260812 03:13:51.829481 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:34447
--local_ip_for_outbound_sockets=127.16.217.66
--tserver_master_addrs=127.16.217.126:43385
--webserver_port=42523
--webserver_interface=127.16.217.66
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:39493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.217.66
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:13:51 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:51 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:13:51.840809 878 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:51.828580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:45327 (local address 127.16.217.126:43385)
0812 03:13:51.829057 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:51.829082 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:51.829468 (+ 386us) server_negotiation.cc:407] Connection header received
0812 03:13:51.831180 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:51.831191 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:51.831459 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:51.831779 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:51.832898 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.833816 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.835009 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.835454 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.835636 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:51.839869 (+ 4233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.65:45327: BlockingRecv error: recv got EOF from 127.16.217.65:45327 (error 108)
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":147,"threads_started":1}
I20260812 03:13:51.840823 879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:51.828782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:45327)
0812 03:13:51.829311 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:51.829352 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:51.830781 (+ 1429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:51.831842 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:51.831850 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:51.832283 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:51.832718 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:51.832731 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.833958 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.833961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:51.834873 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:51.834883 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.835626 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.835630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:51.835820 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:51.836486 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:51.836503 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:51.839609 (+ 3106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":130,"threads_started":1}
W20260812 03:13:51.841646 873 heartbeater.cc:644] Failed to heartbeat to 127.16.217.126:43385 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.217.126:43385: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Aug 12 03:13:51 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:51 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:51.848239 879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:51.842160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:35025)
0812 03:13:51.842353 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:51.842369 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:51.842461 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:51.842717 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:51.842719 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:51.842865 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:51.842980 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:51.842986 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.844054 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.844056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:51.844597 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:51.844605 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.845202 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.845205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:51.845273 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:51.845791 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:51.845802 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:51.848119 (+ 2317us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105}
W20260812 03:13:51.848567 878 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:51.842323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:35025 (local address 127.16.217.126:43385)
0812 03:13:51.842461 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:51.842465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:51.842484 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:13:51.842550 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:51.842553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:51.842605 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:51.842689 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:51.843098 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.843941 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.844718 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.845075 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.845185 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:51.848326 (+ 3141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.65:35025: BlockingRecv error: recv got EOF from 127.16.217.65:35025 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 03:13:51 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:51 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:51.854964 879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:51.848871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:39673)
0812 03:13:51.848968 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:51.848984 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:51.849069 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:51.849315 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:51.849317 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:51.849441 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:51.849513 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:51.849518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.850564 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.850567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:51.851068 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:51.851076 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.851638 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.851641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:51.851701 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:51.852505 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:51.852519 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:51.854852 (+ 2333us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260812 03:13:51.855159 878 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:51.848937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:39673 (local address 127.16.217.126:43385)
0812 03:13:51.849044 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:51.849048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:51.849063 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:13:51.849155 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:51.849158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:51.849201 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:51.849282 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:51.849613 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.850460 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.851182 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:51.851529 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:51.851639 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:51.855042 (+ 3403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.65:39673: BlockingRecv error: recv got EOF from 127.16.217.65:39673 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 03:13:51.855346 873 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:13:51.964145 880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:51.964429 880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:51.964483 880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:51.969395 880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:51.969460 880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:51.969551 880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:13:51.975224 880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:39493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.66.keytab
--principal=oryx/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:34447
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=42523
--tserver_master_addrs=127.16.217.126:43385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:51.976300 880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:51.977366 880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:51.987074 889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:51.987111 880 server_base.cc:1034] running on GCE node
W20260812 03:13:51.987044 888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:51.987062 895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:51.987596 880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:51.988214 880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:51.989379 880 hybrid_clock.cc:648] HybridClock initialized: now 1786504431989366 us; error 26 us; skew 500 ppm
Aug 12 03:13:51 dist-test-slave-7nm7 krb5kdc[32022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504431, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:51.992202 880 init.cc:377] Logged in from keytab as oryx/127.16.217.66@KRBTEST.COM (short username oryx)
I20260812 03:13:51.993043 880 webserver.cc:460] Webserver started at http://127.16.217.66:42523/ using document root <none> and password file <none>
I20260812 03:13:51.993635 880 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:51.993733 880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:51.997735 880 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:13:51.999863 908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:52.001067 880 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:13:52.001210 880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "789ed83eb5ef4930b5268ba9ae50c10f"
format_stamp: "Formatted at 2026-08-12 03:13:50 on dist-test-slave-7nm7"
server_key: "276fd497288ce7371441f665af036079"
server_key_iv: "6c228393a69161f93124fcd88cb29f9c"
server_key_version: "encryptionkey@0"
I20260812 03:13:52.001756 880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:52.015787 880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:52.019294 880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:52.019508 880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:52.020155 880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:52.021009 880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:52.021059 880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:52.021132 880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:52.021171 880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:52.031278 880 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:34447
I20260812 03:13:52.031342 1082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:34447 every 8 connection(s)
I20260812 03:13:52.032379 880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:13:52.037833 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 880
I20260812 03:13:52.038399 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:39065
--local_ip_for_outbound_sockets=127.16.217.67
--tserver_master_addrs=127.16.217.126:43385
--webserver_port=34771
--webserver_interface=127.16.217.67
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:39493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.217.67
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:13:52 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:52 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:13:52.045689 878 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:52.034426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:55569 (local address 127.16.217.126:43385)
0812 03:13:52.034584 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:52.034587 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:52.035440 (+ 853us) server_negotiation.cc:407] Connection header received
0812 03:13:52.037000 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:52.037004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:52.037070 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:52.037192 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:52.038822 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.039373 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.041088 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.041478 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.041609 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:52.045500 (+ 3891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.66:55569: BlockingRecv error: recv got EOF from 127.16.217.66:55569 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:13:52.046440 1089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:52.034803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:55569)
0812 03:13:52.035284 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:52.035326 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:52.036752 (+ 1426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:52.037364 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:52.037370 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:52.037900 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:52.038648 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:52.038673 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.039509 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.039516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:52.040908 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:52.040919 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.041657 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.041661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:52.041809 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:52.042652 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:52.042670 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:52.045293 (+ 2623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":120,"threads_started":1}
W20260812 03:13:52.047096 1084 heartbeater.cc:644] Failed to heartbeat to 127.16.217.126:43385 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.217.126:43385: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Aug 12 03:13:52 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:52 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:52.052757 1089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:52.047601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:37245)
0812 03:13:52.047724 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:52.047739 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:52.047831 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:52.048078 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:52.048081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:52.048250 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:52.048338 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:52.048343 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.049062 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.049067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:52.049619 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:52.049628 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.050079 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.050083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:52.050176 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:52.050735 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:52.050757 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:52.052607 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 03:13:52.052836 878 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:52.047704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:37245 (local address 127.16.217.126:43385)
0812 03:13:52.047834 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:52.047838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:52.047853 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:13:52.047924 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:52.047927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:52.047974 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:52.048062 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:52.048453 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.048950 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.049753 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.049966 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.050077 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:52.052716 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.66:37245: BlockingRecv error: recv got EOF from 127.16.217.66:37245 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 03:13:52 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:52 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:52.058754 1089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:52.053453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:38019)
0812 03:13:52.053547 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:52.053563 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:52.053652 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:52.053894 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:52.053897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:52.054045 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:52.054120 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:52.054125 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.054870 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.054875 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:52.055379 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:52.055386 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.055841 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.055844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:52.055924 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:52.056589 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:52.056615 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:52.058591 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 03:13:52.059018 878 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:52.053536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:38019 (local address 127.16.217.126:43385)
0812 03:13:52.053643 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:52.053647 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:52.053661 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:13:52.053739 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:52.053742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:52.053780 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:52.053875 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:52.054254 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.054754 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.055500 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.055729 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.055839 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:52.058709 (+ 2870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.66:38019: BlockingRecv error: recv got EOF from 127.16.217.66:38019 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 03:13:52.059024 1084 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:13:52.164932 1092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:13:52.165285 1092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:13:52.165367 1092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:13:52.169917 1092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:13:52.170022 1092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:13:52.170118 1092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:13:52.177080 1092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:39493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/krb5kdc/oryx_127.16.217.67.keytab
--principal=oryx/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:39065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=34771
--tserver_master_addrs=127.16.217.126:43385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:13:52.178593 1092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:13:52.179873 1092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:13:52.190299 1098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:13:52.190284 1099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:52.190568 1092 server_base.cc:1034] running on GCE node
W20260812 03:13:52.190299 1105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:13:52.191068 1092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:13:52.191742 1092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:13:52.192956 1092 hybrid_clock.cc:648] HybridClock initialized: now 1786504432192945 us; error 31 us; skew 500 ppm
Aug 12 03:13:52 dist-test-slave-7nm7 krb5kdc[32022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504432, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:13:52.196138 1092 init.cc:377] Logged in from keytab as oryx/127.16.217.67@KRBTEST.COM (short username oryx)
I20260812 03:13:52.197079 1092 webserver.cc:460] Webserver started at http://127.16.217.67:34771/ using document root <none> and password file <none>
I20260812 03:13:52.197672 1092 fs_manager.cc:359] Metadata directory not provided
I20260812 03:13:52.197755 1092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:13:52.201610 1092 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:13:52.203796 1112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:52.205060 1092 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:13:52.205160 1092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "c75b3e08f5d8415a991cabb69c2d51e4"
format_stamp: "Formatted at 2026-08-12 03:13:51 on dist-test-slave-7nm7"
server_key: "e43cfd429611d88873c381f0630d8d9a"
server_key_iv: "335624213a4c2c20f0c1550c6931e37e"
server_key_version: "encryptionkey@0"
I20260812 03:13:52.205680 1092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:13:52.219535 1092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:13:52.222472 1092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:13:52.222640 1092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:13:52.223171 1092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:13:52.223977 1092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:13:52.224027 1092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:52.224068 1092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:13:52.224083 1092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:13:52.233743 1092 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:39065
I20260812 03:13:52.233779 1324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:39065 every 8 connection(s)
I20260812 03:13:52.234787 1092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:13:52.236799 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 1092
Aug 12 03:13:52 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:52 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:13:52.248994 878 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:52.236743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:36793 (local address 127.16.217.126:43385)
0812 03:13:52.236921 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:52.236927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:52.238156 (+ 1229us) server_negotiation.cc:407] Connection header received
0812 03:13:52.239764 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:52.239768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:52.239818 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:52.239909 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:52.241075 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.241768 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.243656 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.243878 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.243962 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:52.248843 (+ 4881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.67:36793: BlockingRecv error: recv got EOF from 127.16.217.67:36793 (error 108)
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":58}
I20260812 03:13:52.249842 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:52.237263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.67:36793)
0812 03:13:52.237760 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:52.237813 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:52.239556 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:52.240080 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:52.240086 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:52.240469 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:52.240943 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:52.240958 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.241902 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.241905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:52.243482 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:52.243493 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.244063 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.244069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:52.244310 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:52.244928 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:52.244951 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:52.248694 (+ 3743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":165,"threads_started":1}
I20260812 03:13:52.250002 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:52.237917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53738 (local address 127.16.217.126:43385)
0812 03:13:52.238205 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:52.238210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:52.238263 (+ 53us) server_negotiation.cc:407] Connection header received
0812 03:13:52.238318 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:52.238322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:52.238378 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:52.238504 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:52.239313 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.239971 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.240865 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.241248 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.241371 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:52.243183 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:13:52.243189 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:13:52.243203 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:13:52.243232 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:13:52.245686 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:52.247780 (+ 2094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:52.247786 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:52.247791 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:52.247927 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:13:52.248289 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:13:52.248292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:13:52.248293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:13:52.248700 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:13:52.248889 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 03:13:52.249599 (+ 710us) server_negotiation.cc:299] Negotiation successful
0812 03:13:52.249828 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":215,"thread_start_us":100,"threads_started":1}
W20260812 03:13:52.250463 1326 heartbeater.cc:644] Failed to heartbeat to 127.16.217.126:43385 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.217.126:43385: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Aug 12 03:13:52 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:52 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:52.256762 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:52.250881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.67:42527)
0812 03:13:52.251090 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:52.251101 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:52.251168 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:52.251439 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:52.251442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:52.251578 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:52.251674 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:52.251679 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.252974 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.252977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:52.253510 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:52.253517 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.254000 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.254003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:52.254097 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:52.254740 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:52.254753 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:52.256645 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128}
W20260812 03:13:52.257196 1334 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:52.250999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:42527 (local address 127.16.217.126:43385)
0812 03:13:52.251122 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:52.251127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:52.251141 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:13:52.251263 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:52.251265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:52.251308 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:52.251393 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:52.252076 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.252798 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.253641 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.253887 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.253997 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:52.256855 (+ 2858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.67:42527: BlockingRecv error: recv got EOF from 127.16.217.67:42527 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 03:13:52 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:52 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:52.263326 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:52.257398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.67:44141)
0812 03:13:52.257610 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:52.257621 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:52.257704 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:52.257975 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:52.257977 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:52.258102 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:52.258174 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:52.258178 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.258935 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.258938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:52.259561 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:52.259571 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.259983 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.259986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:52.260053 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:52.260569 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:52.260580 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:52.263223 (+ 2643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151}
W20260812 03:13:52.263535 1334 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:52.257453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:44141 (local address 127.16.217.126:43385)
0812 03:13:52.257607 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:52.257610 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:52.257685 (+ 75us) server_negotiation.cc:407] Connection header received
0812 03:13:52.257823 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:52.257826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:52.257869 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:52.257944 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:52.258331 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.258817 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.259682 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.259885 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.259971 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:52.263394 (+ 3423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.67:44141: BlockingRecv error: recv got EOF from 127.16.217.67:44141 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260812 03:13:52.263767 1326 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 03:13:52 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:52 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:52.862015 1335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:52.856173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:44391)
0812 03:13:52.856504 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:52.856519 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:52.856622 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:52.856933 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:52.856935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:52.857151 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:52.857261 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:52.857267 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.858173 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.858176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:52.858833 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:52.858843 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.859298 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.859301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:52.859375 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:52.859901 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:52.859912 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:52.861885 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":101,"threads_started":1}
W20260812 03:13:52.862160 1336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:52.856251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:44391 (local address 127.16.217.126:43385)
0812 03:13:52.856565 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:52.856568 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:52.856585 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:13:52.856757 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:52.856760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:52.856810 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:52.856893 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:52.857394 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.857962 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.858957 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:52.859185 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:52.859286 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:52.862020 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.65:44391: BlockingRecv error: recv got EOF from 127.16.217.65:44391 (error 108)
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":108,"threads_started":1}
Aug 12 03:13:53 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:53 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:53.066780 1340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:53.059798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:53083)
0812 03:13:53.060107 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:53.060125 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:53.060248 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:53.060567 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:53.060570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:53.060853 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:53.060965 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:53.060972 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:53.061961 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:53.061964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:53.062622 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:53.062632 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:53.063084 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:53.063086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:53.063150 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:53.063722 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:53.063734 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:53.066652 (+ 2918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":84,"threads_started":1}
W20260812 03:13:53.066918 1336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:53.059986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:53083 (local address 127.16.217.126:43385)
0812 03:13:53.060233 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:53.060238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:53.060268 (+ 30us) server_negotiation.cc:407] Connection header received
0812 03:13:53.060371 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:53.060374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:53.060419 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:53.060491 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:53.061127 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:53.061822 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:53.062752 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:53.062970 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:53.063071 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:53.066791 (+ 3720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.66:53083: BlockingRecv error: recv got EOF from 127.16.217.66:53083 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
Aug 12 03:13:53 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:53 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:53.270697 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:53.264636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.67:50043)
0812 03:13:53.264990 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:53.265007 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:53.265156 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:53.265451 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:53.265454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:53.265686 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:53.265808 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:53.265815 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:53.266788 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:53.266793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:53.267472 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:53.267481 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:53.267976 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:53.267980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:53.268065 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:53.268619 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:53.268631 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:53.270570 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":116,"threads_started":1}
W20260812 03:13:53.270874 1336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:53.264698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:50043 (local address 127.16.217.126:43385)
0812 03:13:53.264888 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:53.264894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:53.265094 (+ 200us) server_negotiation.cc:407] Connection header received
0812 03:13:53.265266 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:53.265270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:53.265326 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:53.265423 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:53.265914 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:53.266657 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:53.267608 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:53.267837 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:53.267969 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:53.270714 (+ 2745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.67:50043: BlockingRecv error: recv got EOF from 127.16.217.67:50043 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 12 03:13:53 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:53 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:53.869007 1344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:53.862991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:34393)
0812 03:13:53.863311 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:53.863325 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:53.863420 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:53.863693 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:53.863695 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:53.863889 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:53.864000 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:53.864006 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:53.864759 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:53.864761 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:53.865305 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:53.865313 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:53.865978 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:53.865981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:53.866077 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:53.866776 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:53.866794 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:53.868850 (+ 2056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":109,"threads_started":1}
W20260812 03:13:53.869156 1345 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:53.863055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:34393 (local address 127.16.217.126:43385)
0812 03:13:53.863354 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:53.863358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:53.863372 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:13:53.863516 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:53.863519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:53.863563 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:53.863645 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:53.864134 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:53.864662 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:53.865500 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:53.865847 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:53.865969 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:53.868996 (+ 3027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.65:34393: BlockingRecv error: recv got EOF from 127.16.217.65:34393 (error 108)
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":117,"threads_started":1}
Aug 12 03:13:54 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:54 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:54.073683 1347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:54.067841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:50731)
0812 03:13:54.068162 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:54.068178 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:54.068282 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:54.068534 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:54.068536 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:54.068728 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:54.068833 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:54.068840 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:54.069828 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:54.069831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:54.070672 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:54.070685 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:54.071127 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:54.071130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:54.071191 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:54.071689 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:54.071701 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:54.073561 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":84,"threads_started":1}
W20260812 03:13:54.073838 1345 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:54.067950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:50731 (local address 127.16.217.126:43385)
0812 03:13:54.068211 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:54.068215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:54.068234 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:13:54.068362 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:54.068364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:54.068410 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:54.068491 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:54.069070 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:54.069681 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:54.070787 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:54.071003 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:54.071086 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:54.073717 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.66:50731: BlockingRecv error: recv got EOF from 127.16.217.66:50731 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 03:13:54 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:54 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:54.277546 1348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:54.271565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.67:46169)
0812 03:13:54.271889 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:54.271903 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:54.272009 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:54.272361 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:54.272364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:54.272552 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:54.272657 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:54.272663 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:54.273437 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:54.273441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:54.274007 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:54.274016 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:54.274563 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:54.274567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:54.274671 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:54.275321 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:54.275333 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:54.277425 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":135,"threads_started":1}
W20260812 03:13:54.277796 1345 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:54.271770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:46169 (local address 127.16.217.126:43385)
0812 03:13:54.271907 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:54.271911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:54.271977 (+ 66us) server_negotiation.cc:407] Connection header received
0812 03:13:54.272145 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:54.272148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:54.272197 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:54.272288 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:54.272780 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:54.273326 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:54.274151 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:54.274415 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:54.274557 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:54.277668 (+ 3111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.67:46169: BlockingRecv error: recv got EOF from 127.16.217.67:46169 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 03:13:54 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:54 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:54.875748 1350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:54.869850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:32977)
0812 03:13:54.870174 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:54.870189 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:54.870336 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:54.870573 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:54.870575 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:54.870778 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:54.870879 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:54.870885 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:54.871628 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:54.871631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:54.872203 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:54.872212 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:54.872892 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:54.872895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:54.873001 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:54.873660 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:54.873680 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:54.875617 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":91,"threads_started":1}
W20260812 03:13:54.875931 1351 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:54.869966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:32977 (local address 127.16.217.126:43385)
0812 03:13:54.870319 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:54.870323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:54.870338 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:13:54.870411 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:54.870413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:54.870456 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:54.870535 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:54.870987 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:54.871519 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:54.872430 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:54.872745 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:54.872884 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:54.875757 (+ 2873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.65:32977: BlockingRecv error: recv got EOF from 127.16.217.65:32977 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":84,"threads_started":1}
Aug 12 03:13:55 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:55 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:13:55.081079 1351 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:55.074644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:34565 (local address 127.16.217.126:43385)
0812 03:13:55.074814 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:55.074819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:55.075122 (+ 303us) server_negotiation.cc:407] Connection header received
0812 03:13:55.075199 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:55.075201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:55.075249 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:55.075318 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:55.075789 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:55.076338 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:55.077200 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:55.077435 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:55.077526 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:55.080936 (+ 3410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.66:34565: BlockingRecv error: recv got EOF from 127.16.217.66:34565 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 03:13:55.081048 1352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:55.074598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:34565)
0812 03:13:55.074963 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:55.074975 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:55.075119 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:55.075359 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:55.075361 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:55.075546 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:55.075651 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:55.075673 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:55.076460 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:55.076463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:55.077029 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:55.077048 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:55.077550 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:55.077553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:55.077619 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:55.078347 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:55.078360 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:55.080655 (+ 2295us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":102,"threads_started":1}
Aug 12 03:13:55 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:55 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:55.287119 1354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:55.278942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.67:55607)
0812 03:13:55.279413 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:55.279429 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:55.279624 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:55.279922 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:55.279925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:55.280211 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:55.280342 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:55.280348 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:55.281354 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:55.281360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:55.282435 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:55.282447 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:55.282977 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:55.282980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:55.283084 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:55.283907 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:55.283926 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:55.286863 (+ 2937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":184,"threads_started":1}
W20260812 03:13:55.287166 1351 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:55.279043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:55607 (local address 127.16.217.126:43385)
0812 03:13:55.279256 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:55.279261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:55.279588 (+ 327us) server_negotiation.cc:407] Connection header received
0812 03:13:55.279725 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:55.279728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:55.279777 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:55.279901 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:55.280466 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:55.281235 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:55.282597 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:55.282844 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:55.282960 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:55.287044 (+ 4084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.67:55607: BlockingRecv error: recv got EOF from 127.16.217.67:55607 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 12 03:13:55 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:55 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:55.884521 1355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:55.876902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:39915)
0812 03:13:55.877384 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:55.877403 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:55.877540 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:55.877906 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:55.877909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:55.878141 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:55.878286 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:55.878292 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:55.879915 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:55.879922 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:55.880694 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:55.880703 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:55.881244 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:55.881248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:55.881332 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:55.882029 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:55.882042 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:55.884394 (+ 2352us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":105,"threads_started":1}
W20260812 03:13:55.884864 1357 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:55.877087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:39915 (local address 127.16.217.126:43385)
0812 03:13:55.877344 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:55.877348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:55.877523 (+ 175us) server_negotiation.cc:407] Connection header received
0812 03:13:55.877714 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:55.877717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:55.877766 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:55.877857 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:55.878530 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:55.879633 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:55.880826 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:55.881086 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:55.881228 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:55.884583 (+ 3355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.65:39915: BlockingRecv error: recv got EOF from 127.16.217.65:39915 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":73,"threads_started":1}
Aug 12 03:13:56 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:56 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:56.088227 1358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:56.082284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:32921)
0812 03:13:56.082677 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:56.082691 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:56.082814 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:56.083074 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:56.083076 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:56.083284 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:56.083390 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:56.083396 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:56.084202 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:56.084205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:56.084777 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:56.084786 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:56.085361 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:56.085364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:56.085491 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:56.086107 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:56.086128 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:56.088103 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":140,"threads_started":1}
W20260812 03:13:56.088361 1357 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:56.082406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:32921 (local address 127.16.217.126:43385)
0812 03:13:56.082567 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:56.082574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:56.082810 (+ 236us) server_negotiation.cc:407] Connection header received
0812 03:13:56.082911 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:56.082913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:56.082962 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:56.083027 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:56.083526 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:56.084075 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:56.084909 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:56.085178 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:56.085344 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:56.088242 (+ 2898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.66:32921: BlockingRecv error: recv got EOF from 127.16.217.66:32921 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 12 03:13:56 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:56 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:56.299669 1361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:56.288229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.67:58421)
0812 03:13:56.288516 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:56.288539 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:56.288648 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:56.288922 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:56.288925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:56.289390 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:56.289638 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:56.289654 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:56.290949 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:56.290952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:56.291660 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:56.291671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:56.292204 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:56.292207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:56.292276 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:56.297304 (+ 5028us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:56.297319 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:56.299533 (+ 2214us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":115,"threads_started":1}
W20260812 03:13:56.299923 1357 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:56.288325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:58421 (local address 127.16.217.126:43385)
0812 03:13:56.288495 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:56.288499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:56.288639 (+ 140us) server_negotiation.cc:407] Connection header received
0812 03:13:56.288742 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:56.288744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:56.288795 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:56.288861 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:56.290016 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:56.290785 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:56.291800 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:56.292045 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:56.292134 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:56.299765 (+ 7631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.67:58421: BlockingRecv error: recv got EOF from 127.16.217.67:58421 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 12 03:13:56 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:56 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:56.891805 1362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:56.885613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:56049)
0812 03:13:56.885908 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:56.885923 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:56.886028 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:56.886288 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:56.886290 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:56.886747 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:56.886854 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:56.886861 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:56.887652 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:56.887654 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:56.888272 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:56.888281 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:56.888765 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:56.888768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:56.888827 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:56.889352 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:56.889364 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:56.891630 (+ 2266us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":116,"threads_started":1}
W20260812 03:13:56.891999 1363 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:56.885662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:56049 (local address 127.16.217.126:43385)
0812 03:13:56.885941 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:56.885946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:56.885992 (+ 46us) server_negotiation.cc:407] Connection header received
0812 03:13:56.886106 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:56.886109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:56.886156 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:56.886264 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:56.886960 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:56.887536 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:56.888404 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:56.888657 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:56.888751 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:56.891804 (+ 3053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.65:56049: BlockingRecv error: recv got EOF from 127.16.217.65:56049 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":70,"threads_started":1}
Aug 12 03:13:57 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:57 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:57.095945 1366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:57.089190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:35589)
0812 03:13:57.089548 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:57.089564 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:57.089731 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:57.090020 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:57.090023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:57.090320 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:57.090427 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:57.090433 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:57.091222 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:57.091227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:57.091813 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:57.091821 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:57.092480 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:57.092484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:57.092596 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:57.093314 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:57.093348 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:57.095816 (+ 2468us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":116,"threads_started":1}
W20260812 03:13:57.096136 1363 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:57.089346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:35589 (local address 127.16.217.126:43385)
0812 03:13:57.089536 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:57.089541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:57.089656 (+ 115us) server_negotiation.cc:407] Connection header received
0812 03:13:57.089835 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:57.089840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:57.089899 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:57.089991 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:57.090547 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:57.091099 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:57.092070 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:57.092342 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:57.092471 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:57.095965 (+ 3494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.66:35589: BlockingRecv error: recv got EOF from 127.16.217.66:35589 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 12 03:13:57 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:57 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:57.306947 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:57.300657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.67:56045)
0812 03:13:57.300940 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:57.300963 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:57.301064 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:57.301384 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:57.301387 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:57.301658 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:57.301766 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:57.301772 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:57.302821 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:57.302826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:57.303414 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:57.303423 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:57.303897 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:57.303901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:57.303985 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:57.304512 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:57.304525 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:57.306768 (+ 2243us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":102,"threads_started":1}
W20260812 03:13:57.306993 1363 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:57.300827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:56045 (local address 127.16.217.126:43385)
0812 03:13:57.300965 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:57.300969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:57.301038 (+ 69us) server_negotiation.cc:407] Connection header received
0812 03:13:57.301148 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:57.301150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:57.301206 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:57.301303 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:57.301916 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:57.302709 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:57.303541 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:57.303778 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:57.303892 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:57.306890 (+ 2998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.67:56045: BlockingRecv error: recv got EOF from 127.16.217.67:56045 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 03:13:57 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:57 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:57.899202 1369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:57.892744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:58893)
0812 03:13:57.893018 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:57.893031 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:57.893154 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:57.893424 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:57.893426 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:57.893635 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:57.893737 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:57.893743 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:57.894663 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:57.894667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:57.895311 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:57.895322 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:57.895906 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:57.895910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:57.895977 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:57.896631 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:57.896645 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:57.898885 (+ 2240us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":90,"threads_started":1}
W20260812 03:13:57.899290 1371 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:57.892821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:58893 (local address 127.16.217.126:43385)
0812 03:13:57.893159 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:57.893163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:57.893210 (+ 47us) server_negotiation.cc:407] Connection header received
0812 03:13:57.893251 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:57.893253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:57.893301 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:57.893386 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:57.893869 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:57.894536 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:57.895458 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:57.895690 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:57.895887 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:57.899020 (+ 3133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.65:58893: BlockingRecv error: recv got EOF from 127.16.217.65:58893 (error 108)
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":151,"threads_started":1}
Aug 12 03:13:58 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:58 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:58.102555 1372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:58.096847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:46975)
0812 03:13:58.097166 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:58.097183 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:58.097339 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:58.097620 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:58.097623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:58.097839 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:58.097934 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:58.097940 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:58.098763 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:58.098768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:58.099324 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:58.099332 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:58.099793 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:58.099797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:58.099882 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:58.100450 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:58.100462 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:58.102366 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":108,"threads_started":1}
W20260812 03:13:58.102620 1371 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:58.096927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:46975 (local address 127.16.217.126:43385)
0812 03:13:58.097141 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:58.097145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:58.097271 (+ 126us) server_negotiation.cc:407] Connection header received
0812 03:13:58.097440 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:58.097444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:58.097496 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:58.097583 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:58.098057 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:58.098656 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:58.099447 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:58.099674 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:58.099778 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:58.102505 (+ 2727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.66:46975: BlockingRecv error: recv got EOF from 127.16.217.66:46975 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 12 03:13:58 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:58 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:58.314296 1374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:58.307782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.67:50489)
0812 03:13:58.308141 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:58.308154 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:58.308308 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:58.308578 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:58.308581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:58.308809 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:58.308903 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:58.308908 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:58.309751 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:58.309756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:58.310538 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:58.310562 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:58.311206 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:58.311210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:58.311290 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:58.311924 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:58.311939 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:58.314086 (+ 2147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":114,"threads_started":1}
W20260812 03:13:58.314363 1371 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:58.307914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:50489 (local address 127.16.217.126:43385)
0812 03:13:58.308084 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:58.308089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:58.308240 (+ 151us) server_negotiation.cc:407] Connection header received
0812 03:13:58.308416 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:58.308420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:58.308473 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:58.308556 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:58.309044 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:58.309632 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:58.310785 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:58.311063 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:58.311194 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:58.314219 (+ 3025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.67:50489: BlockingRecv error: recv got EOF from 127.16.217.67:50489 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 12 03:13:58 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:58 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:58.906451 1376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:58.900443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:34733)
0812 03:13:58.900818 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:58.900833 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:58.900942 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:58.901229 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:58.901231 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:58.901428 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:58.901526 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:58.901531 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:58.902445 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:58.902449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:58.903275 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:58.903286 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:58.903720 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:58.903723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:58.903783 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:58.904302 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:58.904314 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:58.906299 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":99,"threads_started":1}
W20260812 03:13:58.906602 1377 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:58.900538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:34733 (local address 127.16.217.126:43385)
0812 03:13:58.900811 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:58.900815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:58.900906 (+ 91us) server_negotiation.cc:407] Connection header received
0812 03:13:58.901023 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:58.901026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:58.901072 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:58.901162 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:58.901660 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:58.902334 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:58.903391 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:58.903618 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:58.903701 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:58.906465 (+ 2764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.65:34733: BlockingRecv error: recv got EOF from 127.16.217.65:34733 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":82,"threads_started":1}
Aug 12 03:13:59 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:59 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:59.109100 1378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:59.103525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:37417)
0812 03:13:59.103863 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:59.103877 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:59.103978 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:59.104225 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:59.104227 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:59.104411 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:59.104515 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:59.104521 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:59.105403 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:59.105407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:59.106048 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:59.106059 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:59.106512 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:59.106515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:59.106575 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:59.107135 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:59.107146 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:59.108984 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":112,"threads_started":1}
W20260812 03:13:59.109344 1377 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:59.103591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:37417 (local address 127.16.217.126:43385)
0812 03:13:59.103767 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:59.103772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:59.103979 (+ 207us) server_negotiation.cc:407] Connection header received
0812 03:13:59.104051 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:59.104053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:59.104106 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:59.104185 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:59.104646 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:59.105277 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:59.106156 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:59.106418 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:59.106505 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:59.109174 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.66:37417: BlockingRecv error: recv got EOF from 127.16.217.66:37417 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 12 03:13:59 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:59 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:59.320673 1379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:59.315144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.67:54469)
0812 03:13:59.315479 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:59.315492 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:59.315633 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:59.315893 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:59.315916 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:59.316130 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:59.316219 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:59.316224 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:59.316975 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:59.316980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:59.317536 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:59.317544 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:59.318026 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:59.318030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:59.318117 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:59.318672 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:59.318683 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:59.320555 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":102,"threads_started":1}
W20260812 03:13:59.320915 1377 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:59.315214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:54469 (local address 127.16.217.126:43385)
0812 03:13:59.315377 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:59.315382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:59.315568 (+ 186us) server_negotiation.cc:407] Connection header received
0812 03:13:59.315730 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:59.315734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:59.315783 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:59.315871 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:59.316337 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:59.316868 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:59.317664 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:59.317906 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:59.318013 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:59.320687 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.67:54469: BlockingRecv error: recv got EOF from 127.16.217.67:54469 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 03:13:59 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:13:59 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:13:59.912833 1382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:13:59.907199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:60281)
0812 03:13:59.907523 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 03:13:59.907537 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:13:59.907629 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:13:59.907865 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:13:59.907868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:13:59.908067 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:13:59.908170 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:59.908175 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:59.908990 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:59.908992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:59.909663 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:13:59.909673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:13:59.910149 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:13:59.910152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:13:59.910222 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:59.910841 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:13:59.910854 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:13:59.912683 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":147,"threads_started":1}
W20260812 03:13:59.913213 1385 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:13:59.907387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:60281 (local address 127.16.217.126:43385)
0812 03:13:59.907613 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 03:13:59.907617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:13:59.907631 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:13:59.907711 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:13:59.907714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:13:59.907755 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:13:59.907833 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:13:59.908274 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:59.908892 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:59.909771 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:13:59.910000 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:13:59.910100 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:13:59.912825 (+ 2725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.65:60281: BlockingRecv error: recv got EOF from 127.16.217.65:60281 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":64,"threads_started":1}
Aug 12 03:14:00 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:00 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:00.116531 1386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:00.109859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:34307)
0812 03:14:00.110259 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:00.110273 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:00.110380 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:00.110617 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:00.110619 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:00.110813 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:00.110913 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:00.110918 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:00.112138 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:00.112145 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:00.113087 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:00.113109 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:00.113746 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:00.113750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:00.113853 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:00.114558 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:00.114576 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:00.116405 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":183,"threads_started":1}
W20260812 03:14:00.116739 1385 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:00.110002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:34307 (local address 127.16.217.126:43385)
0812 03:14:00.110144 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:00.110149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:00.110365 (+ 216us) server_negotiation.cc:407] Connection header received
0812 03:14:00.110449 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:00.110451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:00.110500 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:00.110597 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:00.111019 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:00.111747 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:00.113314 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:00.113563 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:00.113664 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:00.116591 (+ 2927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.66:34307: BlockingRecv error: recv got EOF from 127.16.217.66:34307 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 12 03:14:00 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:00 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:00.329598 1387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:00.321808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.67:35725)
0812 03:14:00.322572 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:00.322594 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:00.322877 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:00.323237 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:00.323240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:00.323432 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:00.323529 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:00.323534 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:00.324423 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:00.324427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:00.325049 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:00.325058 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:00.325770 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:00.325773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:00.325841 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:00.326690 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:00.326702 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:00.329412 (+ 2710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":121,"threads_started":1}
W20260812 03:14:00.329685 1385 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:00.321874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:35725 (local address 127.16.217.126:43385)
0812 03:14:00.322010 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:00.322014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:00.322668 (+ 654us) server_negotiation.cc:407] Connection header received
0812 03:14:00.323002 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:00.323006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:00.323069 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:00.323152 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:00.323740 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:00.324296 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:00.325324 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:00.325544 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:00.325768 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:00.329563 (+ 3795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.67:35725: BlockingRecv error: recv got EOF from 127.16.217.67:35725 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 03:14:00 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:00 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:00.920863 1392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:00.913848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:33505)
0812 03:14:00.914151 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:00.914165 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:00.914313 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:00.914626 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:00.914628 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:00.914837 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:00.914937 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:00.914943 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:00.915737 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:00.915741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:00.916304 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:00.916312 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:00.916782 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:00.916785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:00.916873 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:00.917560 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:00.917572 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:00.920605 (+ 3033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":109,"threads_started":1}
W20260812 03:14:00.921048 1393 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:00.913914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:33505 (local address 127.16.217.126:43385)
0812 03:14:00.914211 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:00.914214 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:00.914260 (+ 46us) server_negotiation.cc:407] Connection header received
0812 03:14:00.914430 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:00.914435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:00.914502 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:00.914596 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:00.915062 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:00.915620 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:00.916425 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:00.916663 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:00.916774 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:00.920864 (+ 4090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.65:33505: BlockingRecv error: recv got EOF from 127.16.217.65:33505 (error 108)
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":150,"threads_started":1}
Aug 12 03:14:01 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:01 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:01.123193 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:01.117346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:36499)
0812 03:14:01.117650 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:01.117665 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:01.117770 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:01.117981 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:01.117983 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:01.118169 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:01.118329 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:01.118336 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:01.119139 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:01.119144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:01.119723 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:01.119732 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:01.120376 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:01.120380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:01.120471 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:01.121062 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:01.121074 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:01.123019 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":99,"threads_started":1}
W20260812 03:14:01.123277 1393 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:01.117416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:36499 (local address 127.16.217.126:43385)
0812 03:14:01.117570 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:01.117577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:01.117723 (+ 146us) server_negotiation.cc:407] Connection header received
0812 03:14:01.117838 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:01.117840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:01.117884 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:01.117951 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:01.118445 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:01.119017 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:01.119852 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:01.120248 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:01.120374 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:01.123153 (+ 2779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.66:36499: BlockingRecv error: recv got EOF from 127.16.217.66:36499 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 03:14:01 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:01 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:01.339922 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:01.330631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.67:48833)
0812 03:14:01.330967 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:01.330982 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:01.331129 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:01.331434 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:01.331436 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:01.331634 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:01.331733 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:01.331738 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:01.332536 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:01.332538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:01.333124 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:01.333132 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:01.333622 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:01.333625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:01.333693 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:01.337141 (+ 3448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:01.337154 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:01.339705 (+ 2551us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":124,"threads_started":1}
W20260812 03:14:01.340027 1393 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:01.330699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:48833 (local address 127.16.217.126:43385)
0812 03:14:01.330869 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:01.330874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:01.331080 (+ 206us) server_negotiation.cc:407] Connection header received
0812 03:14:01.331231 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:01.331234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:01.331305 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:01.331379 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:01.331878 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:01.332425 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:01.333255 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:01.333492 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:01.333579 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:01.339885 (+ 6306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.67:48833: BlockingRecv error: recv got EOF from 127.16.217.67:48833 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 12 03:14:01 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:01 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:01.928016 1396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:01.921915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:56609)
0812 03:14:01.922279 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:01.922294 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:01.922391 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:01.922645 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:01.922647 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:01.922844 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:01.922950 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:01.922998 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:01.923748 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:01.923751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:01.924353 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:01.924363 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:01.924772 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:01.924775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:01.924838 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:01.925437 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:01.925451 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:01.927890 (+ 2439us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":76,"threads_started":1}
W20260812 03:14:01.928362 1398 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:01.921979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:56609 (local address 127.16.217.126:43385)
0812 03:14:01.922325 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:01.922329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:01.922345 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:14:01.922480 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:01.922482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:01.922526 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:01.922604 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:01.923123 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:01.923644 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:01.924460 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:01.924671 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:01.924756 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:01.928139 (+ 3383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.65:56609: BlockingRecv error: recv got EOF from 127.16.217.65:56609 (error 108)
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":130,"threads_started":1}
Aug 12 03:14:02 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:02 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:02.130502 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:02.124069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:59229)
0812 03:14:02.124352 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:02.124366 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:02.124458 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:02.124713 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:02.124715 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:02.124896 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:02.124992 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:02.124997 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:02.125987 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:02.125990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:02.126698 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:02.126710 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:02.127348 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:02.127357 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:02.127429 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:02.128043 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:02.128081 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:02.130375 (+ 2294us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":92,"threads_started":1}
W20260812 03:14:02.130709 1398 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:02.124228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:59229 (local address 127.16.217.126:43385)
0812 03:14:02.124377 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:02.124380 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:02.124418 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:14:02.124544 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:02.124546 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:02.124589 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:02.124672 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:02.125215 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:02.125848 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:02.126825 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:02.127047 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:02.127160 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:02.130531 (+ 3371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.66:59229: BlockingRecv error: recv got EOF from 127.16.217.66:59229 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 03:14:02 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:02 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:02.346988 1402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:02.340992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.67:34689)
0812 03:14:02.341379 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:02.341398 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:02.341511 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:02.341761 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:02.341763 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:02.341940 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:02.342034 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:02.342045 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:02.342943 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:02.342949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:02.343790 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:02.343805 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:02.344337 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:02.344340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:02.344397 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:02.344891 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:02.344902 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:02.346867 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":117,"threads_started":1}
W20260812 03:14:02.347162 1398 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:02.341140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:34689 (local address 127.16.217.126:43385)
0812 03:14:02.341336 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:02.341339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:02.341473 (+ 134us) server_negotiation.cc:407] Connection header received
0812 03:14:02.341601 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:02.341603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:02.341646 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:02.341714 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:02.342175 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:02.342772 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:02.343925 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:02.344201 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:02.344323 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:02.346999 (+ 2676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.67:34689: BlockingRecv error: recv got EOF from 127.16.217.67:34689 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 12 03:14:02 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:02 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:02.934831 1404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:02.929072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:51631)
0812 03:14:02.929411 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:02.929425 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:02.929530 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:02.929842 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:02.929844 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:02.930049 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:02.930173 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:02.930179 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:02.930998 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:02.931001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:02.931576 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:02.931585 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:02.932012 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:02.932015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:02.932093 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:02.932681 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:02.932694 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:02.934709 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":111,"threads_started":1}
W20260812 03:14:02.935029 1405 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:02.929180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:51631 (local address 127.16.217.126:43385)
0812 03:14:02.929439 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:02.929443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:02.929472 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:14:02.929624 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:02.929627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:02.929671 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:02.929752 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:02.930349 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:02.930895 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:02.931690 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:02.931912 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:02.932002 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:02.934871 (+ 2869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.65:51631: BlockingRecv error: recv got EOF from 127.16.217.65:51631 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":78,"threads_started":1}
Aug 12 03:14:03 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:03 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:03.137398 1408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:03.131384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:50611)
0812 03:14:03.131680 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:03.131694 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:03.131791 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:03.132107 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:03.132110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:03.132580 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:03.132685 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:03.132692 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:03.133540 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:03.133544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:03.134193 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:03.134204 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:03.134752 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:03.134755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:03.134834 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:03.135401 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:03.135414 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:03.137254 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":117,"threads_started":1}
W20260812 03:14:03.137607 1405 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:03.131512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:50611 (local address 127.16.217.126:43385)
0812 03:14:03.131730 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:03.131734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:03.131752 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:14:03.131909 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:03.131911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:03.131959 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:03.132033 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:03.132823 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:03.133393 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:03.134402 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:03.134629 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:03.134710 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:03.137426 (+ 2716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.66:50611: BlockingRecv error: recv got EOF from 127.16.217.66:50611 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 03:14:03 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:03 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:14:03.355250 1405 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:03.347976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:54131 (local address 127.16.217.126:43385)
0812 03:14:03.348147 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:03.348156 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:03.348302 (+ 146us) server_negotiation.cc:407] Connection header received
0812 03:14:03.348372 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:03.348374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:03.348424 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:03.348496 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:03.348926 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:03.349487 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:03.350641 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:03.350871 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:03.350963 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:03.355137 (+ 4174us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.67:54131: BlockingRecv error: recv got EOF from 127.16.217.67:54131 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:14:03.355343 1409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:03.347852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.67:54131)
0812 03:14:03.348181 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:03.348195 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:03.348299 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:03.348530 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:03.348532 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:03.348719 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:03.348815 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:03.348820 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:03.349607 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:03.349610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:03.350504 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:03.350515 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:03.350999 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:03.351002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:03.351072 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:03.351879 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:03.351892 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:03.354972 (+ 3080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":120,"threads_started":1}
Aug 12 03:14:03 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:03 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:03.941138 1411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:03.935645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:35365)
0812 03:14:03.935916 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:03.935930 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:03.936020 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:03.936387 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:03.936390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:03.936580 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:03.936692 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:03.936698 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:03.937465 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:03.937468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:03.938039 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:03.938047 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:03.938601 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:03.938604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:03.938686 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:03.939268 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:03.939281 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:03.941010 (+ 1729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":97,"threads_started":1}
W20260812 03:14:03.941350 1412 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:03.935741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:35365 (local address 127.16.217.126:43385)
0812 03:14:03.935982 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:03.935986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:03.936000 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:14:03.936165 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:03.936168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:03.936218 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:03.936324 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:03.936801 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:03.937364 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:03.938272 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:03.938497 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:03.938589 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:03.941166 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.65:35365: BlockingRecv error: recv got EOF from 127.16.217.65:35365 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":86,"threads_started":1}
Aug 12 03:14:04 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:04 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:04.144100 1413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:04.138209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:45367)
0812 03:14:04.138582 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:04.138595 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:04.138705 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:04.138939 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:04.138941 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:04.139134 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:04.139234 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:04.139240 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:04.139994 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:04.139996 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:04.140546 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:04.140554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:04.140974 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:04.140976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:04.141042 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:04.141799 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:04.141812 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:04.143928 (+ 2116us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":132,"threads_started":1}
W20260812 03:14:04.144493 1412 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:04.138320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:45367 (local address 127.16.217.126:43385)
0812 03:14:04.138464 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:04.138468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:04.138675 (+ 207us) server_negotiation.cc:407] Connection header received
0812 03:14:04.138782 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:04.138784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:04.138830 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:04.138907 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:04.139355 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:04.139893 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:04.140652 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:04.140867 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:04.140951 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:04.144115 (+ 3164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.66:45367: BlockingRecv error: recv got EOF from 127.16.217.66:45367 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 03:14:04 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:04 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:04.361809 1415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:04.356208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.67:57327)
0812 03:14:04.356486 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:04.356502 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:04.356616 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:04.356865 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:04.356868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:04.357058 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:04.357158 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:04.357163 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:04.357913 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:04.357916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:04.358512 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:04.358520 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:04.358970 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:04.358972 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:04.359032 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:04.359600 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:04.359619 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:04.361690 (+ 2071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":100,"threads_started":1}
W20260812 03:14:04.362010 1412 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:04.356317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:57327 (local address 127.16.217.126:43385)
0812 03:14:04.356449 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:04.356452 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:04.356579 (+ 127us) server_negotiation.cc:407] Connection header received
0812 03:14:04.356700 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:04.356702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:04.356748 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:04.356824 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:04.357275 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:04.357798 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:04.358628 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:04.358858 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:04.358946 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:04.361876 (+ 2930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.67:57327: BlockingRecv error: recv got EOF from 127.16.217.67:57327 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 03:14:04 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:04 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:04.947916 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:04.942010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:49513)
0812 03:14:04.942335 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:04.942349 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:04.942439 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:04.942814 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:04.942818 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:04.943077 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:04.943177 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:04.943183 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:04.944223 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:04.944225 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:04.944789 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:04.944797 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:04.945275 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:04.945278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:04.945344 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:04.945892 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:04.945905 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:04.947786 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":116,"threads_started":1}
W20260812 03:14:04.948127 1420 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:04.942107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:49513 (local address 127.16.217.126:43385)
0812 03:14:04.942369 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:04.942373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:04.942390 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:14:04.942546 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:04.942550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:04.942602 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:04.942727 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:04.943351 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:04.944122 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:04.944915 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:04.945157 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:04.945251 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:04.947942 (+ 2691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.65:49513: BlockingRecv error: recv got EOF from 127.16.217.65:49513 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":120,"threads_started":1}
Aug 12 03:14:05 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:05 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:05.151417 1421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:05.145049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:50063)
0812 03:14:05.145453 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:05.145469 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:05.145583 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:05.145906 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:05.145909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:05.146131 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:05.146283 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:05.146291 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:05.147235 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:05.147238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:05.148010 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:05.148021 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:05.148586 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:05.148589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:05.148661 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:05.149199 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:05.149211 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:05.151297 (+ 2086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":142,"threads_started":1}
W20260812 03:14:05.151705 1420 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:05.145210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:50063 (local address 127.16.217.126:43385)
0812 03:14:05.145399 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:05.145402 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:05.145546 (+ 144us) server_negotiation.cc:407] Connection header received
0812 03:14:05.145719 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:05.145722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:05.145771 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:05.145849 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:05.146424 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:05.147108 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:05.148178 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:05.148483 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:05.148573 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:05.151571 (+ 2998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.66:50063: BlockingRecv error: recv got EOF from 127.16.217.66:50063 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 12 03:14:05 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:05 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:05.368460 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:05.362629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.67:42115)
0812 03:14:05.362942 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:05.362962 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:05.363099 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:05.363364 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:05.363367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:05.363588 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:05.363688 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:05.363694 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:05.364458 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:05.364463 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:05.365032 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:05.365039 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:05.365563 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:05.365567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:05.365669 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:05.366266 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:05.366278 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:05.368304 (+ 2026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":112,"threads_started":1}
W20260812 03:14:05.368566 1420 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:05.362725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:42115 (local address 127.16.217.126:43385)
0812 03:14:05.362865 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:05.362870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:05.363036 (+ 166us) server_negotiation.cc:407] Connection header received
0812 03:14:05.363204 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:05.363207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:05.363258 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:05.363345 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:05.363802 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:05.364350 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:05.365156 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:05.365450 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:05.365557 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:05.368439 (+ 2882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.67:42115: BlockingRecv error: recv got EOF from 127.16.217.67:42115 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 03:14:05 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:05 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:05.954725 1425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:05.948770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:41083)
0812 03:14:05.949043 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:05.949057 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:05.949144 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:05.949442 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:05.949444 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:05.949618 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:05.949723 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:05.949729 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:05.950707 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:05.950711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:05.951513 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:05.951522 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:05.951987 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:05.951991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:05.952082 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:05.952663 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:05.952676 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:05.954601 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":95,"threads_started":1}
W20260812 03:14:05.954859 1427 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:05.948895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:41083 (local address 127.16.217.126:43385)
0812 03:14:05.949224 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:05.949228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:05.949241 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:14:05.949279 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:05.949282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:05.949320 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:05.949403 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:05.949849 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:05.950563 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:05.951618 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:05.951852 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:05.951935 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:05.954725 (+ 2790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.65:41083: BlockingRecv error: recv got EOF from 127.16.217.65:41083 (error 108)
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":103,"threads_started":1}
Aug 12 03:14:06 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:06 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:06.162848 1428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:06.152341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:35541)
0812 03:14:06.152660 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:06.152675 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:06.152817 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:06.153070 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:06.153073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:06.153299 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:06.153392 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:06.153398 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:06.154151 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:06.154162 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:06.154790 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:06.154799 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:06.155243 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:06.155246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:06.155328 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:06.160605 (+ 5277us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:06.160617 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:06.162657 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":86,"threads_started":1}
W20260812 03:14:06.162930 1427 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:06.152448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:35541 (local address 127.16.217.126:43385)
0812 03:14:06.152621 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:06.152625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:06.152751 (+ 126us) server_negotiation.cc:407] Connection header received
0812 03:14:06.152913 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:06.152916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:06.152965 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:06.153044 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:06.153514 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:06.154043 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:06.154909 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:06.155130 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:06.155235 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:06.162804 (+ 7569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.66:35541: BlockingRecv error: recv got EOF from 127.16.217.66:35541 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 12 03:14:06 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:06 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.67@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:06.380065 1431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:06.369343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.67:60399)
0812 03:14:06.369683 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:06.369701 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:06.369855 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:06.370136 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:06.370140 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:06.370384 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:06.370478 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:06.370484 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:06.371388 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:06.371390 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:06.372248 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:06.372262 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:06.372718 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:06.372720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:06.372779 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:06.377837 (+ 5058us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:06.377849 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:06.379859 (+ 2010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":94,"threads_started":1}
W20260812 03:14:06.380091 1427 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:06.369469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:60399 (local address 127.16.217.126:43385)
0812 03:14:06.369662 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:06.369667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:06.369791 (+ 124us) server_negotiation.cc:407] Connection header received
0812 03:14:06.369959 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:06.369963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:06.370019 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:06.370106 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:06.370601 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:06.371245 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:06.372375 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:06.372613 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:06.372701 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:06.379986 (+ 7285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.67:60399: BlockingRecv error: recv got EOF from 127.16.217.67:60399 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 12 03:14:06 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:06 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.65@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:06.961606 1432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:06.955953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.65:51211)
0812 03:14:06.956274 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:06.956293 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:06.956402 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:06.956719 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:06.956725 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:06.956977 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:06.957111 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:06.957118 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:06.957970 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:06.957972 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:06.958545 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:06.958554 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:06.958977 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:06.958980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:06.959039 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:06.959590 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:06.959602 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:06.961485 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":101,"threads_started":1}
W20260812 03:14:06.961819 1433 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:06.956083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:51211 (local address 127.16.217.126:43385)
0812 03:14:06.956435 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:06.956440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:06.956455 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:14:06.956502 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:06.956504 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:06.956544 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:06.956655 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:06.957241 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:06.957871 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:06.958648 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:06.958877 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:06.958959 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:06.961629 (+ 2670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.65:51211: BlockingRecv error: recv got EOF from 127.16.217.65:51211 (error 108)
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":108,"threads_started":1}
Aug 12 03:14:07 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:14:07 dist-test-slave-7nm7 krb5kdc[32022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.217.66@KRBTEST.COM for oryx/127.16.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:14:07.169387 1435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:07.163945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43385 (local address 127.16.217.66:50851)
0812 03:14:07.164265 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:07.164278 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:07.164373 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:07.164619 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:07.164621 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:07.164821 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:07.164930 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:07.164936 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:07.165763 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:07.165765 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:07.166408 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:07.166418 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:07.166871 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:07.166874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:07.166931 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:07.167485 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:07.167496 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:07.169272 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.217.126:43385: Server oryx/127.16.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":118,"threads_started":1}
W20260812 03:14:07.169615 1433 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:07.164004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:50851 (local address 127.16.217.126:43385)
0812 03:14:07.164167 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:07.164172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:07.164364 (+ 192us) server_negotiation.cc:407] Connection header received
0812 03:14:07.164456 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:07.164458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:07.164506 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:07.164586 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:07.165068 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:07.165656 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:07.166541 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:07.166767 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:07.166864 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:07.169448 (+ 2584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.217.66:50851: BlockingRecv error: recv got EOF from 127.16.217.66:50851 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:14:07.240324 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 609
I20260812 03:14:07.253002 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 880
I20260812 03:14:07.287791 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 1092
I20260812 03:14:07.307317 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 487
2026-08-12T03:14:07Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18523 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T03:14:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:14:07Z Disabled control of system clock
I20260812 03:14:07.349524 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:34247
--webserver_interface=127.16.217.126
--webserver_port=0
--builtin_ntp_servers=127.16.217.84:43997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:34247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 03:14:07.480347 1443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:14:07.480638 1443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:14:07.480696 1443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:14:07.484892 1443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:14:07.484956 1443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:14:07.484972 1443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:14:07.485011 1443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:14:07.485044 1443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:14:07.489908 1443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:34247
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:34247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:14:07.491077 1443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:14:07.492491 1443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:14:07.501178 1453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:07.501178 1456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:07.501221 1451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:14:07.501379 1443 server_base.cc:1034] running on GCE node
I20260812 03:14:07.502053 1443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:14:07.503216 1443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:14:07.504400 1443 hybrid_clock.cc:648] HybridClock initialized: now 1786504447504372 us; error 47 us; skew 500 ppm
I20260812 03:14:07.506983 1443 webserver.cc:460] Webserver started at http://127.16.217.126:42863/ using document root <none> and password file <none>
I20260812 03:14:07.507642 1443 fs_manager.cc:359] Metadata directory not provided
I20260812 03:14:07.507714 1443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:14:07.508002 1443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:14:07.510655 1443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "7e8fb6fad68e4c3aa7784dc7fd9b6229"
format_stamp: "Formatted at 2026-08-12 03:14:07 on dist-test-slave-7nm7"
server_key: "f5e0e052474dde2103d3e036ad216260"
server_key_iv: "80e2f8d5decfd0583cfca478c996fe83"
server_key_version: "encryptionkey@0"
I20260812 03:14:07.511260 1443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "7e8fb6fad68e4c3aa7784dc7fd9b6229"
format_stamp: "Formatted at 2026-08-12 03:14:07 on dist-test-slave-7nm7"
server_key: "f5e0e052474dde2103d3e036ad216260"
server_key_iv: "80e2f8d5decfd0583cfca478c996fe83"
server_key_version: "encryptionkey@0"
I20260812 03:14:07.515085 1443 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:14:07.517779 1466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:07.519209 1443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:14:07.519347 1443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "7e8fb6fad68e4c3aa7784dc7fd9b6229"
format_stamp: "Formatted at 2026-08-12 03:14:07 on dist-test-slave-7nm7"
server_key: "f5e0e052474dde2103d3e036ad216260"
server_key_iv: "80e2f8d5decfd0583cfca478c996fe83"
server_key_version: "encryptionkey@0"
I20260812 03:14:07.519461 1443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:14:07.530615 1443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:14:07.531281 1443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:14:07.531476 1443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:14:07.538606 1443 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:34247
I20260812 03:14:07.538661 1548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:34247 every 8 connection(s)
I20260812 03:14:07.539788 1443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:14:07.543246 1551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:14:07.546307 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 1443
I20260812 03:14:07.546432 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:14:07.546741 17253 external_mini_cluster.cc:1428] Setting key dfcaca786d67f40b29f9ca1c870b484a
I20260812 03:14:07.549608 1551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229: Bootstrap starting.
I20260812 03:14:07.552012 1551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229: Neither blocks nor log segments found. Creating new log.
I20260812 03:14:07.552804 1551 log.cc:824] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229: Log is configured to *not* fsync() on all Append() calls
I20260812 03:14:07.554795 1551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229: No bootstrap required, opened a new log
I20260812 03:14:07.556313 1554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:07.548255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52278 (local address 127.16.217.126:34247)
0812 03:14:07.548956 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:07.548972 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:07.549012 (+ 40us) server_negotiation.cc:407] Connection header received
0812 03:14:07.549810 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:07.549835 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:07.550404 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:07.550733 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:07.551190 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:07.552175 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:07.553075 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:07.553550 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:07.553726 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:07.554600 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:14:07.554606 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:14:07.554619 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:14:07.555050 (+ 431us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:14:07.555104 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:14:07.555125 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:14:07.555232 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:07.555346 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 03:14:07.555594 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":146,"threads_started":1}
I20260812 03:14:07.558923 1551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8fb6fad68e4c3aa7784dc7fd9b6229" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 34247 } }
I20260812 03:14:07.559187 1551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:14:07.559275 1551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e8fb6fad68e4c3aa7784dc7fd9b6229, State: Initialized, Role: FOLLOWER
I20260812 03:14:07.559733 1551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8fb6fad68e4c3aa7784dc7fd9b6229" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 34247 } }
I20260812 03:14:07.559883 1551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:14:07.559965 1551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:14:07.560063 1551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:14:07.561184 1551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8fb6fad68e4c3aa7784dc7fd9b6229" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 34247 } }
I20260812 03:14:07.561554 1551 leader_election.cc:302] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e8fb6fad68e4c3aa7784dc7fd9b6229; no voters:
I20260812 03:14:07.561908 1551 leader_election.cc:288] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:14:07.562058 1556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:14:07.562403 1556 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229 [term 1 LEADER]: Becoming Leader. State: Replica: 7e8fb6fad68e4c3aa7784dc7fd9b6229, State: Running, Role: LEADER
I20260812 03:14:07.562711 1556 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8fb6fad68e4c3aa7784dc7fd9b6229" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 34247 } }
I20260812 03:14:07.563439 1551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:14:07.564931 1557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e8fb6fad68e4c3aa7784dc7fd9b6229" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8fb6fad68e4c3aa7784dc7fd9b6229" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 34247 } } }
I20260812 03:14:07.565204 1557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229 [sys.catalog]: This master's current role is: LEADER
I20260812 03:14:07.565637 1558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e8fb6fad68e4c3aa7784dc7fd9b6229. Latest consensus state: current_term: 1 leader_uuid: "7e8fb6fad68e4c3aa7784dc7fd9b6229" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8fb6fad68e4c3aa7784dc7fd9b6229" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 34247 } } }
I20260812 03:14:07.565814 1558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229 [sys.catalog]: This master's current role is: LEADER
I20260812 03:14:07.566388 1583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:14:07.569761 1583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:14:07.575284 1583 catalog_manager.cc:1347] Generated new cluster ID: 63bd87eb6b064d908932d7be85b9a0e5
I20260812 03:14:07.575358 1583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:14:07.589887 1583 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:14:07.591305 1583 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:14:07.597752 1583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e8fb6fad68e4c3aa7784dc7fd9b6229: Generated new TSK 0
I20260812 03:14:07.598531 1583 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:14:07.603536 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:34247
--builtin_ntp_servers=127.16.217.84:43997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 03:14:07.725690 1600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:14:07.725952 1600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:14:07.725989 1600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:14:07.730381 1600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:14:07.730449 1600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:14:07.730504 1600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:14:07.735532 1600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:34247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:14:07.736662 1600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:14:07.738183 1600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:14:07.748413 1613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:07.748488 1610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:07.748482 1611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:14:07.748764 1600 server_base.cc:1034] running on GCE node
I20260812 03:14:07.749476 1600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:14:07.750129 1600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:14:07.751319 1600 hybrid_clock.cc:648] HybridClock initialized: now 1786504447751308 us; error 25 us; skew 500 ppm
I20260812 03:14:07.753070 1600 webserver.cc:460] Webserver started at http://127.16.217.65:33091/ using document root <none> and password file <none>
I20260812 03:14:07.753671 1600 fs_manager.cc:359] Metadata directory not provided
I20260812 03:14:07.753739 1600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:14:07.753965 1600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:14:07.756074 1600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "0c079ea7c88b443cba98a545a4ae0cf4"
format_stamp: "Formatted at 2026-08-12 03:14:07 on dist-test-slave-7nm7"
server_key: "67c2bc3bfb712c4abd6f5ced3402a332"
server_key_iv: "7fa1073897c3a7ad8d0d1b00da0a431a"
server_key_version: "encryptionkey@0"
I20260812 03:14:07.756656 1600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "0c079ea7c88b443cba98a545a4ae0cf4"
format_stamp: "Formatted at 2026-08-12 03:14:07 on dist-test-slave-7nm7"
server_key: "67c2bc3bfb712c4abd6f5ced3402a332"
server_key_iv: "7fa1073897c3a7ad8d0d1b00da0a431a"
server_key_version: "encryptionkey@0"
I20260812 03:14:07.760609 1600 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:14:07.763479 1621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:07.764874 1600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:14:07.765017 1600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "0c079ea7c88b443cba98a545a4ae0cf4"
format_stamp: "Formatted at 2026-08-12 03:14:07 on dist-test-slave-7nm7"
server_key: "67c2bc3bfb712c4abd6f5ced3402a332"
server_key_iv: "7fa1073897c3a7ad8d0d1b00da0a431a"
server_key_version: "encryptionkey@0"
I20260812 03:14:07.765159 1600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:14:07.801275 1600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:14:07.802065 1600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:14:07.802330 1600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:14:07.803252 1600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:14:07.804446 1600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:14:07.804543 1600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:07.804621 1600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:14:07.804658 1600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:07.814384 1600 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:45667
I20260812 03:14:07.814420 1822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:45667 every 8 connection(s)
I20260812 03:14:07.815343 1600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:14:07.823436 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 1600
I20260812 03:14:07.823576 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:14:07.823874 17253 external_mini_cluster.cc:1428] Setting key 4de89611d15b0660974576c71e288918
I20260812 03:14:07.825220 1554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:07.817294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:53533 (local address 127.16.217.126:34247)
0812 03:14:07.817465 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:07.817469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:07.818600 (+ 1131us) server_negotiation.cc:407] Connection header received
0812 03:14:07.820102 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:07.820105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:07.820162 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:07.820277 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:07.821158 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:07.821699 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:07.823006 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:07.823303 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:07.823421 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:07.824704 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:14:07.824720 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:14:07.824723 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:14:07.824739 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:14:07.824761 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:14:07.824771 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:14:07.824852 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:07.825009 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 03:14:07.825064 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 03:14:07.825982 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:34247
--builtin_ntp_servers=127.16.217.84:43997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 03:14:07.826136 1826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:07.817682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:34247 (local address 127.16.217.65:53533)
0812 03:14:07.818433 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:07.818474 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:07.819873 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:07.820449 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:07.820457 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:07.820536 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:07.821009 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:07.821020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:07.821842 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:07.821845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:07.822879 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:07.822890 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:07.823462 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:07.823467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:07.823708 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:07.824351 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:14:07.824352 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:07.824391 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:14:07.824493 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:14:07.824865 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:14:07.824872 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:14:07.824878 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:14:07.824981 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 03:14:07.825285 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":120,"threads_started":1}
I20260812 03:14:07.827421 1823 heartbeater.cc:347] Connected to a master server at 127.16.217.126:34247
I20260812 03:14:07.827682 1823 heartbeater.cc:464] Registering TS with master...
I20260812 03:14:07.828223 1823 heartbeater.cc:511] Master 127.16.217.126:34247 requested a full tablet report, sending...
I20260812 03:14:07.829782 1497 ts_manager.cc:194] Registered new tserver with Master: 0c079ea7c88b443cba98a545a4ae0cf4 (127.16.217.65:45667)
I20260812 03:14:07.831019 1497 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.217.65:53533
W20260812 03:14:07.951005 1829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:14:07.951279 1829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:14:07.951321 1829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:14:07.955755 1829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:14:07.955829 1829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:14:07.955888 1829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:14:07.962132 1829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:34247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:14:07.963366 1829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:14:07.965047 1829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:14:07.975628 1840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:07.975708 1844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:07.975649 1837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:14:07.976063 1829 server_base.cc:1034] running on GCE node
I20260812 03:14:07.976614 1829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:14:07.977313 1829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:14:07.978499 1829 hybrid_clock.cc:648] HybridClock initialized: now 1786504447978471 us; error 44 us; skew 500 ppm
I20260812 03:14:07.980439 1829 webserver.cc:460] Webserver started at http://127.16.217.66:36551/ using document root <none> and password file <none>
I20260812 03:14:07.981042 1829 fs_manager.cc:359] Metadata directory not provided
I20260812 03:14:07.981105 1829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:14:07.981345 1829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:14:07.983476 1829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "a72b7ae55be44ddb902cad8d5e34dc37"
format_stamp: "Formatted at 2026-08-12 03:14:07 on dist-test-slave-7nm7"
server_key: "d614d936a65597936411d3032e2ca440"
server_key_iv: "515cb77472bd65e20e5aa188f8401b60"
server_key_version: "encryptionkey@0"
I20260812 03:14:07.984054 1829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "a72b7ae55be44ddb902cad8d5e34dc37"
format_stamp: "Formatted at 2026-08-12 03:14:07 on dist-test-slave-7nm7"
server_key: "d614d936a65597936411d3032e2ca440"
server_key_iv: "515cb77472bd65e20e5aa188f8401b60"
server_key_version: "encryptionkey@0"
I20260812 03:14:07.987735 1829 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:14:07.990334 1856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:07.991472 1829 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:14:07.991621 1829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "a72b7ae55be44ddb902cad8d5e34dc37"
format_stamp: "Formatted at 2026-08-12 03:14:07 on dist-test-slave-7nm7"
server_key: "d614d936a65597936411d3032e2ca440"
server_key_iv: "515cb77472bd65e20e5aa188f8401b60"
server_key_version: "encryptionkey@0"
I20260812 03:14:07.991742 1829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:14:08.005323 1829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:14:08.006003 1829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:14:08.006204 1829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:14:08.006811 1829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:14:08.007759 1829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:14:08.007840 1829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:08.007912 1829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:14:08.007957 1829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:08.016687 1829 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:37591
I20260812 03:14:08.016709 2051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:37591 every 8 connection(s)
I20260812 03:14:08.017695 1829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:14:08.024724 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 1829
I20260812 03:14:08.024816 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:14:08.025115 17253 external_mini_cluster.cc:1428] Setting key fc3ef31c8c7fbdb94e3bf92904068e6a
I20260812 03:14:08.027323 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:34247
--builtin_ntp_servers=127.16.217.84:43997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 03:14:08.027377 1554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:08.019462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:33065 (local address 127.16.217.126:34247)
0812 03:14:08.019620 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:08.019624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:08.020844 (+ 1220us) server_negotiation.cc:407] Connection header received
0812 03:14:08.022201 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:08.022204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:08.022308 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:08.022384 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:08.023383 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:08.023919 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:08.025179 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:08.025475 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:08.025575 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:08.026869 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:14:08.026873 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:14:08.026876 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:14:08.026888 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:14:08.026909 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:14:08.026918 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:14:08.026996 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:08.027141 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 03:14:08.027225 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:14:08.028184 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:08.019848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:34247 (local address 127.16.217.66:33065)
0812 03:14:08.020697 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:08.020737 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:08.022002 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:08.022604 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:08.022612 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:08.022708 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:08.023257 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:08.023272 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:08.024018 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:08.024021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:08.025042 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:08.025053 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:08.025650 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:08.025658 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:08.025853 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:08.026586 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:14:08.026589 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:08.026642 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:14:08.026724 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:14:08.027004 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:14:08.027010 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:14:08.027015 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:14:08.027124 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 03:14:08.027331 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":121,"threads_started":1}
I20260812 03:14:08.029282 2052 heartbeater.cc:347] Connected to a master server at 127.16.217.126:34247
I20260812 03:14:08.029527 2052 heartbeater.cc:464] Registering TS with master...
I20260812 03:14:08.030035 2052 heartbeater.cc:511] Master 127.16.217.126:34247 requested a full tablet report, sending...
I20260812 03:14:08.031112 1497 ts_manager.cc:194] Registered new tserver with Master: a72b7ae55be44ddb902cad8d5e34dc37 (127.16.217.66:37591)
I20260812 03:14:08.031685 1497 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.217.66:33065
W20260812 03:14:08.143534 2056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:14:08.143797 2056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:14:08.143836 2056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:14:08.147814 2056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:14:08.147867 2056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:14:08.147919 2056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:14:08.152513 2056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:34247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:14:08.153529 2056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:14:08.154943 2056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:14:08.164603 2068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:08.164587 2065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:08.164587 2064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:14:08.164844 2056 server_base.cc:1034] running on GCE node
I20260812 03:14:08.165364 2056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:14:08.166031 2056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:14:08.167220 2056 hybrid_clock.cc:648] HybridClock initialized: now 1786504448167205 us; error 26 us; skew 500 ppm
I20260812 03:14:08.169167 2056 webserver.cc:460] Webserver started at http://127.16.217.67:34077/ using document root <none> and password file <none>
I20260812 03:14:08.169759 2056 fs_manager.cc:359] Metadata directory not provided
I20260812 03:14:08.169852 2056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:14:08.170086 2056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:14:08.172204 2056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "d37dcfe157514dd09305f6493882111d"
format_stamp: "Formatted at 2026-08-12 03:14:08 on dist-test-slave-7nm7"
server_key: "6fae3e52664039f0cc461537eba6d023"
server_key_iv: "2ad326a2b2f68fb3f988a715c2a282ca"
server_key_version: "encryptionkey@0"
I20260812 03:14:08.172783 2056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "d37dcfe157514dd09305f6493882111d"
format_stamp: "Formatted at 2026-08-12 03:14:08 on dist-test-slave-7nm7"
server_key: "6fae3e52664039f0cc461537eba6d023"
server_key_iv: "2ad326a2b2f68fb3f988a715c2a282ca"
server_key_version: "encryptionkey@0"
I20260812 03:14:08.176671 2056 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:14:08.179338 2078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:08.180646 2056 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:14:08.180786 2056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "d37dcfe157514dd09305f6493882111d"
format_stamp: "Formatted at 2026-08-12 03:14:08 on dist-test-slave-7nm7"
server_key: "6fae3e52664039f0cc461537eba6d023"
server_key_iv: "2ad326a2b2f68fb3f988a715c2a282ca"
server_key_version: "encryptionkey@0"
I20260812 03:14:08.181131 2056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:14:08.194331 2056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:14:08.195123 2056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:14:08.195339 2056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:14:08.195945 2056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:14:08.197124 2056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:14:08.197206 2056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:08.197278 2056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:14:08.197320 2056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:08.207262 2056 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:37869
I20260812 03:14:08.207293 2288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:37869 every 8 connection(s)
I20260812 03:14:08.208191 2056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:14:08.215099 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 2056
I20260812 03:14:08.215242 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:14:08.215567 17253 external_mini_cluster.cc:1428] Setting key 458414784c6a13dae66c3f1dc18cfa09
I20260812 03:14:08.217828 1554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:08.210046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:56881 (local address 127.16.217.126:34247)
0812 03:14:08.210265 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:08.210270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:08.211353 (+ 1083us) server_negotiation.cc:407] Connection header received
0812 03:14:08.212696 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:08.212699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:08.212746 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:08.212825 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:08.213703 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:08.214299 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:08.215506 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:08.215824 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:08.215941 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:08.217341 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:14:08.217345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:14:08.217347 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:14:08.217358 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:14:08.217376 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:14:08.217384 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:14:08.217444 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:08.217620 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 03:14:08.217668 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 03:14:08.218727 2291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:08.210504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:34247 (local address 127.16.217.67:56881)
0812 03:14:08.211205 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:08.211246 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:08.212503 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:08.212990 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:08.212997 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:08.213084 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:08.213550 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:08.213562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:08.214424 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:08.214427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:08.215385 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:08.215395 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:08.216040 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:08.216046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:08.216278 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:08.217061 (+ 783us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:14:08.217063 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:08.217102 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:14:08.217201 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:14:08.217471 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:14:08.217479 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:14:08.217485 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:14:08.217628 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 03:14:08.217918 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":120,"threads_started":1}
I20260812 03:14:08.219983 2289 heartbeater.cc:347] Connected to a master server at 127.16.217.126:34247
I20260812 03:14:08.220242 2289 heartbeater.cc:464] Registering TS with master...
I20260812 03:14:08.220768 2289 heartbeater.cc:511] Master 127.16.217.126:34247 requested a full tablet report, sending...
I20260812 03:14:08.221804 1497 ts_manager.cc:194] Registered new tserver with Master: d37dcfe157514dd09305f6493882111d (127.16.217.67:37869)
I20260812 03:14:08.222456 1497 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.217.67:56881
I20260812 03:14:08.229772 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 03:14:08.233392 1554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:08.232402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52294 (local address 127.16.217.126:34247)
0812 03:14:08.232557 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:08.232564 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:08.232581 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:14:08.232755 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:08.232775 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:08.232829 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:08.232937 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:08.233254 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52294: BlockingRecv error: recv got EOF from 127.0.0.1:52294 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:14:08.233587 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 1600
I20260812 03:14:08.269913 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 1829
I20260812 03:14:08.290124 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 2056
I20260812 03:14:08.305306 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 1443
2026-08-12T03:14:08Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (995 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T03:14:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:14:08Z Disabled control of system clock
I20260812 03:14:08.345220 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:42653
--webserver_interface=127.16.217.126
--webserver_port=0
--builtin_ntp_servers=127.16.217.84:38313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:42653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 03:14:08.470422 2302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:14:08.470703 2302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:14:08.470741 2302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:14:08.474776 2302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:14:08.474834 2302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:14:08.474849 2302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:14:08.474865 2302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:14:08.474881 2302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:14:08.479628 2302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:38313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:42653
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:42653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:14:08.480692 2302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:14:08.482180 2302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:14:08.490970 2312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:14:08.491191 2302 server_base.cc:1034] running on GCE node
W20260812 03:14:08.490970 2311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:08.490970 2316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:14:08.491961 2302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:14:08.493259 2302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:14:08.494459 2302 hybrid_clock.cc:648] HybridClock initialized: now 1786504448494427 us; error 43 us; skew 500 ppm
I20260812 03:14:08.496407 2302 webserver.cc:460] Webserver started at http://127.16.217.126:42665/ using document root <none> and password file <none>
I20260812 03:14:08.497021 2302 fs_manager.cc:359] Metadata directory not provided
I20260812 03:14:08.497116 2302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:14:08.497359 2302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:14:08.499514 2302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "b47aaa5a31ef4d6da2ebfd387560b078"
format_stamp: "Formatted at 2026-08-12 03:14:08 on dist-test-slave-7nm7"
server_key: "cf9feda7cf5aaa25d7a8a8da0804eb14"
server_key_iv: "950bf0a79a4a4b6d0f03de2b43143612"
server_key_version: "encryptionkey@0"
I20260812 03:14:08.500093 2302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "b47aaa5a31ef4d6da2ebfd387560b078"
format_stamp: "Formatted at 2026-08-12 03:14:08 on dist-test-slave-7nm7"
server_key: "cf9feda7cf5aaa25d7a8a8da0804eb14"
server_key_iv: "950bf0a79a4a4b6d0f03de2b43143612"
server_key_version: "encryptionkey@0"
I20260812 03:14:08.503933 2302 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:14:08.506456 2323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:08.507736 2302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 03:14:08.507884 2302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "b47aaa5a31ef4d6da2ebfd387560b078"
format_stamp: "Formatted at 2026-08-12 03:14:08 on dist-test-slave-7nm7"
server_key: "cf9feda7cf5aaa25d7a8a8da0804eb14"
server_key_iv: "950bf0a79a4a4b6d0f03de2b43143612"
server_key_version: "encryptionkey@0"
I20260812 03:14:08.508005 2302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:14:08.513301 2302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:14:08.513967 2302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:14:08.514180 2302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:14:08.521983 2302 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:42653
I20260812 03:14:08.521996 2412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:42653 every 8 connection(s)
I20260812 03:14:08.523056 2302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:14:08.526707 2413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:14:08.532367 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 2302
I20260812 03:14:08.532529 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:14:08.532634 2413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078: Bootstrap starting.
I20260812 03:14:08.532874 17253 external_mini_cluster.cc:1428] Setting key e5b5c78de570800ffd8282f0222ec13e
I20260812 03:14:08.535327 2413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078: Neither blocks nor log segments found. Creating new log.
I20260812 03:14:08.536095 2413 log.cc:824] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078: Log is configured to *not* fsync() on all Append() calls
I20260812 03:14:08.538421 2413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078: No bootstrap required, opened a new log
I20260812 03:14:08.538995 2418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:08.534346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38978 (local address 127.16.217.126:42653)
0812 03:14:08.535064 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:08.535082 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:08.535118 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:14:08.535806 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:08.535824 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:08.536176 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:08.536580 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:08.537490 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:14:08.537496 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:14:08.537510 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:14:08.537522 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:14:08.537592 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:14:08.537620 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:14:08.537737 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:08.537811 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 03:14:08.538064 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":145,"threads_started":1}
I20260812 03:14:08.542551 2413 raft_consensus.cc:348] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b47aaa5a31ef4d6da2ebfd387560b078" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 42653 } }
I20260812 03:14:08.542802 2413 raft_consensus.cc:374] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:14:08.542894 2413 raft_consensus.cc:729] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b47aaa5a31ef4d6da2ebfd387560b078, State: Initialized, Role: FOLLOWER
I20260812 03:14:08.543457 2413 consensus_queue.cc:260] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b47aaa5a31ef4d6da2ebfd387560b078" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 42653 } }
I20260812 03:14:08.543622 2413 raft_consensus.cc:388] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:14:08.543673 2413 raft_consensus.cc:482] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:14:08.543748 2413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:14:08.544698 2413 raft_consensus.cc:504] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b47aaa5a31ef4d6da2ebfd387560b078" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 42653 } }
I20260812 03:14:08.545025 2413 leader_election.cc:302] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b47aaa5a31ef4d6da2ebfd387560b078; no voters:
I20260812 03:14:08.545346 2413 leader_election.cc:288] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:14:08.545455 2420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:14:08.545725 2420 raft_consensus.cc:686] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078 [term 1 LEADER]: Becoming Leader. State: Replica: b47aaa5a31ef4d6da2ebfd387560b078, State: Running, Role: LEADER
I20260812 03:14:08.546060 2420 consensus_queue.cc:237] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b47aaa5a31ef4d6da2ebfd387560b078" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 42653 } }
I20260812 03:14:08.546721 2413 sys_catalog.cc:564] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:14:08.547806 2422 sys_catalog.cc:455] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b47aaa5a31ef4d6da2ebfd387560b078. Latest consensus state: current_term: 1 leader_uuid: "b47aaa5a31ef4d6da2ebfd387560b078" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b47aaa5a31ef4d6da2ebfd387560b078" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 42653 } } }
I20260812 03:14:08.547932 2422 sys_catalog.cc:458] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078 [sys.catalog]: This master's current role is: LEADER
I20260812 03:14:08.547866 2421 sys_catalog.cc:455] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b47aaa5a31ef4d6da2ebfd387560b078" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b47aaa5a31ef4d6da2ebfd387560b078" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 42653 } } }
I20260812 03:14:08.548907 2421 sys_catalog.cc:458] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078 [sys.catalog]: This master's current role is: LEADER
I20260812 03:14:08.549000 2434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:14:08.551839 2434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:14:08.557190 2434 catalog_manager.cc:1347] Generated new cluster ID: 82b17941283141f28905ae07e2b8245b
I20260812 03:14:08.557268 2434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:14:08.577227 2434 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:14:08.577606 2434 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:14:08.583217 2434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b47aaa5a31ef4d6da2ebfd387560b078: Generated new TSK 0
I20260812 03:14:08.583984 2434 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:14:08.594736 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:42653
--builtin_ntp_servers=127.16.217.84:38313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 03:14:08.709759 2451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:14:08.710058 2451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:14:08.710130 2451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:14:08.714290 2451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:14:08.714372 2451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:14:08.714461 2451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:14:08.719223 2451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:38313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:42653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:14:08.720363 2451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:14:08.721768 2451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:14:08.731927 2467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:08.731923 2457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:08.731951 2459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:14:08.732146 2451 server_base.cc:1034] running on GCE node
I20260812 03:14:08.732758 2451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:14:08.733457 2451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:14:08.734629 2451 hybrid_clock.cc:648] HybridClock initialized: now 1786504448734613 us; error 29 us; skew 500 ppm
I20260812 03:14:08.736857 2451 webserver.cc:460] Webserver started at http://127.16.217.65:32925/ using document root <none> and password file <none>
I20260812 03:14:08.737476 2451 fs_manager.cc:359] Metadata directory not provided
I20260812 03:14:08.737608 2451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:14:08.737826 2451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:14:08.739992 2451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "a47c1d4d40b34b1e9dfdfdad063a0929"
format_stamp: "Formatted at 2026-08-12 03:14:08 on dist-test-slave-7nm7"
server_key: "65be839390411370843dcabe64a15f6a"
server_key_iv: "268302143f1691442b0abf99bd80da2d"
server_key_version: "encryptionkey@0"
I20260812 03:14:08.740567 2451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "a47c1d4d40b34b1e9dfdfdad063a0929"
format_stamp: "Formatted at 2026-08-12 03:14:08 on dist-test-slave-7nm7"
server_key: "65be839390411370843dcabe64a15f6a"
server_key_iv: "268302143f1691442b0abf99bd80da2d"
server_key_version: "encryptionkey@0"
I20260812 03:14:08.744411 2451 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260812 03:14:08.747295 2476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:08.748725 2451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:14:08.748847 2451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "a47c1d4d40b34b1e9dfdfdad063a0929"
format_stamp: "Formatted at 2026-08-12 03:14:08 on dist-test-slave-7nm7"
server_key: "65be839390411370843dcabe64a15f6a"
server_key_iv: "268302143f1691442b0abf99bd80da2d"
server_key_version: "encryptionkey@0"
I20260812 03:14:08.748980 2451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:14:08.753670 2451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:14:08.754381 2451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:14:08.754567 2451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:14:08.755156 2451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:14:08.756139 2451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:14:08.756211 2451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:08.756278 2451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:14:08.756325 2451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:08.766126 2451 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:43857
I20260812 03:14:08.766201 2677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:43857 every 8 connection(s)
I20260812 03:14:08.767163 2451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:14:08.772850 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 2451
I20260812 03:14:08.772989 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:14:08.773356 17253 external_mini_cluster.cc:1428] Setting key 4f94a9b9ba6b395aae17e0944e8b7540
I20260812 03:14:08.773882 2418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:08.769447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:58611 (local address 127.16.217.126:42653)
0812 03:14:08.769646 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:08.769650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:08.770708 (+ 1058us) server_negotiation.cc:407] Connection header received
0812 03:14:08.772042 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:08.772046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:08.772089 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:08.772167 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:08.773392 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:14:08.773395 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:14:08.773398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:14:08.773418 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:14:08.773455 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:14:08.773467 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:14:08.773564 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:08.773697 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 03:14:08.773746 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 03:14:08.775494 2680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:08.769726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:42653 (local address 127.16.217.65:58611)
0812 03:14:08.770550 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:08.770592 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:08.771831 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:08.772328 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:08.772334 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:08.772420 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:08.773136 (+ 716us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:14:08.773138 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:08.773178 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:14:08.773246 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:14:08.773538 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:14:08.773548 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:14:08.773554 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:14:08.773742 (+ 188us) client_negotiation.cc:240] Negotiation successful
0812 03:14:08.773979 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":125,"threads_started":1}
I20260812 03:14:08.776183 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:42653
--builtin_ntp_servers=127.16.217.84:38313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 03:14:08.776757 2678 heartbeater.cc:347] Connected to a master server at 127.16.217.126:42653
I20260812 03:14:08.777014 2678 heartbeater.cc:464] Registering TS with master...
I20260812 03:14:08.777441 2678 heartbeater.cc:511] Master 127.16.217.126:42653 requested a full tablet report, sending...
I20260812 03:14:08.778981 2352 ts_manager.cc:194] Registered new tserver with Master: a47c1d4d40b34b1e9dfdfdad063a0929 (127.16.217.65:43857)
W20260812 03:14:08.888640 2681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:14:08.888899 2681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:14:08.888942 2681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:14:08.893100 2681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:14:08.893162 2681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:14:08.893221 2681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:14:08.898134 2681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:38313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:42653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:14:08.899379 2681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:14:08.900847 2681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:14:08.911856 2691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:08.911873 2688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:08.911856 2693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:14:08.912238 2681 server_base.cc:1034] running on GCE node
I20260812 03:14:08.913143 2681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:14:08.914330 2681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:14:08.915596 2681 hybrid_clock.cc:648] HybridClock initialized: now 1786504448915573 us; error 52 us; skew 500 ppm
I20260812 03:14:08.917922 2681 webserver.cc:460] Webserver started at http://127.16.217.66:35725/ using document root <none> and password file <none>
I20260812 03:14:08.918654 2681 fs_manager.cc:359] Metadata directory not provided
I20260812 03:14:08.918752 2681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:14:08.919010 2681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:14:08.921243 2681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "7189bb7909324af8b2e6514048efb631"
format_stamp: "Formatted at 2026-08-12 03:14:08 on dist-test-slave-7nm7"
server_key: "fe2cf9b41dfb3e5158073170c4ca15f5"
server_key_iv: "a7759b7565555c883e99812dd0a9fa15"
server_key_version: "encryptionkey@0"
I20260812 03:14:08.921862 2681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "7189bb7909324af8b2e6514048efb631"
format_stamp: "Formatted at 2026-08-12 03:14:08 on dist-test-slave-7nm7"
server_key: "fe2cf9b41dfb3e5158073170c4ca15f5"
server_key_iv: "a7759b7565555c883e99812dd0a9fa15"
server_key_version: "encryptionkey@0"
I20260812 03:14:08.926031 2681 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 03:14:08.929411 2703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:08.931174 2681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:14:08.931546 2681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "7189bb7909324af8b2e6514048efb631"
format_stamp: "Formatted at 2026-08-12 03:14:08 on dist-test-slave-7nm7"
server_key: "fe2cf9b41dfb3e5158073170c4ca15f5"
server_key_iv: "a7759b7565555c883e99812dd0a9fa15"
server_key_version: "encryptionkey@0"
I20260812 03:14:08.931730 2681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:14:08.937292 2681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:14:08.937937 2681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:14:08.938169 2681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:14:08.938769 2681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:14:08.939746 2681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:14:08.939827 2681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:08.939903 2681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:14:08.939945 2681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:08.949666 2681 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:45831
I20260812 03:14:08.949699 2899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:45831 every 8 connection(s)
I20260812 03:14:08.950654 2681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:14:08.953464 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 2681
I20260812 03:14:08.953619 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:14:08.953948 17253 external_mini_cluster.cc:1428] Setting key d406d39e37d1147b722d1b5aeee03fdf
I20260812 03:14:08.956408 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:42653
--builtin_ntp_servers=127.16.217.84:38313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 03:14:08.957521 2418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:08.952612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:52301 (local address 127.16.217.126:42653)
0812 03:14:08.952756 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:08.952761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:08.954099 (+ 1338us) server_negotiation.cc:407] Connection header received
0812 03:14:08.955605 (+ 1506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:08.955609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:08.955647 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:08.955746 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:08.957061 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:14:08.957065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:14:08.957067 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:14:08.957079 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:14:08.957097 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:14:08.957105 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:14:08.957187 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:08.957374 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 03:14:08.957414 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:14:08.958796 2906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:08.953156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:42653 (local address 127.16.217.66:52301)
0812 03:14:08.953927 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:08.953977 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:08.955354 (+ 1377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:08.955938 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:08.955947 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:08.956056 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:08.956739 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:14:08.956743 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:08.956798 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:14:08.956887 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:14:08.957209 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:14:08.957221 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:14:08.957230 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 03:14:08.957405 (+ 175us) client_negotiation.cc:240] Negotiation successful
0812 03:14:08.957648 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":134,"threads_started":1}
I20260812 03:14:08.960089 2900 heartbeater.cc:347] Connected to a master server at 127.16.217.126:42653
I20260812 03:14:08.960337 2900 heartbeater.cc:464] Registering TS with master...
I20260812 03:14:08.960832 2900 heartbeater.cc:511] Master 127.16.217.126:42653 requested a full tablet report, sending...
I20260812 03:14:08.962095 2352 ts_manager.cc:194] Registered new tserver with Master: 7189bb7909324af8b2e6514048efb631 (127.16.217.66:45831)
W20260812 03:14:09.075773 2907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:14:09.076040 2907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:14:09.076081 2907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:14:09.080217 2907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:14:09.080276 2907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:14:09.080334 2907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:14:09.085181 2907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:38313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:42653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:14:09.086382 2907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:14:09.087880 2907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:14:09.097827 2913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:09.097846 2918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:14:09.098001 2907 server_base.cc:1034] running on GCE node
W20260812 03:14:09.098312 2915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:14:09.098809 2907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:14:09.099468 2907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:14:09.100656 2907 hybrid_clock.cc:648] HybridClock initialized: now 1786504449100614 us; error 55 us; skew 500 ppm
I20260812 03:14:09.102805 2907 webserver.cc:460] Webserver started at http://127.16.217.67:39821/ using document root <none> and password file <none>
I20260812 03:14:09.103380 2907 fs_manager.cc:359] Metadata directory not provided
I20260812 03:14:09.103442 2907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:14:09.103623 2907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:14:09.105690 2907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "f2af786b45c04d5998c8d2a32699408d"
format_stamp: "Formatted at 2026-08-12 03:14:09 on dist-test-slave-7nm7"
server_key: "7209b3366b183ede90c89e704213f0ae"
server_key_iv: "dc2bdfd66258c45aeb88fe5935fcdda8"
server_key_version: "encryptionkey@0"
I20260812 03:14:09.106223 2907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "f2af786b45c04d5998c8d2a32699408d"
format_stamp: "Formatted at 2026-08-12 03:14:09 on dist-test-slave-7nm7"
server_key: "7209b3366b183ede90c89e704213f0ae"
server_key_iv: "dc2bdfd66258c45aeb88fe5935fcdda8"
server_key_version: "encryptionkey@0"
I20260812 03:14:09.109980 2907 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:14:09.113325 2935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:09.114701 2907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:14:09.114809 2907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "f2af786b45c04d5998c8d2a32699408d"
format_stamp: "Formatted at 2026-08-12 03:14:09 on dist-test-slave-7nm7"
server_key: "7209b3366b183ede90c89e704213f0ae"
server_key_iv: "dc2bdfd66258c45aeb88fe5935fcdda8"
server_key_version: "encryptionkey@0"
I20260812 03:14:09.114899 2907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:14:09.119928 2907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:14:09.120507 2907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:14:09.120651 2907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:14:09.121168 2907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:14:09.122072 2907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:14:09.122120 2907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:09.122159 2907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:14:09.122174 2907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:09.131945 2907 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:40833
I20260812 03:14:09.132030 3133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:40833 every 8 connection(s)
I20260812 03:14:09.132964 2907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:14:09.135092 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 2907
I20260812 03:14:09.135222 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:14:09.135701 17253 external_mini_cluster.cc:1428] Setting key 5823991c413214f4bae2b45a6839da84
I20260812 03:14:09.140614 2418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:09.135254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:49017 (local address 127.16.217.126:42653)
0812 03:14:09.135413 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:09.135421 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:09.137277 (+ 1856us) server_negotiation.cc:407] Connection header received
0812 03:14:09.138902 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:09.138906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:09.138950 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:09.139035 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:09.140119 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:14:09.140123 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:14:09.140126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:14:09.140138 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:14:09.140159 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:14:09.140175 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:14:09.140270 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:09.140434 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 03:14:09.140479 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:14:09.141484 3140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:09.135930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:42653 (local address 127.16.217.67:49017)
0812 03:14:09.137100 (+ 1170us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:09.137161 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:09.138700 (+ 1539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:09.139205 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:09.139212 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:09.139300 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:09.139858 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:14:09.139860 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:09.139900 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:14:09.139976 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:14:09.140268 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:14:09.140279 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:14:09.140288 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 03:14:09.140470 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 03:14:09.140716 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":218,"threads_started":1}
I20260812 03:14:09.142648 3135 heartbeater.cc:347] Connected to a master server at 127.16.217.126:42653
I20260812 03:14:09.142863 3135 heartbeater.cc:464] Registering TS with master...
I20260812 03:14:09.143268 3135 heartbeater.cc:511] Master 127.16.217.126:42653 requested a full tablet report, sending...
I20260812 03:14:09.144294 2352 ts_manager.cc:194] Registered new tserver with Master: f2af786b45c04d5998c8d2a32699408d (127.16.217.67:40833)
I20260812 03:14:09.150017 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 03:14:09.153282 2418 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:09.152087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38990 (local address 127.16.217.126:42653)
0812 03:14:09.152257 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:09.152263 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:09.152424 (+ 161us) server_negotiation.cc:407] Connection header received
0812 03:14:09.152642 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:09.152652 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:09.152689 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:09.152780 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:09.153111 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38990: BlockingRecv error: recv got EOF from 127.0.0.1:38990 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 03:14:09.153496 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 2451
I20260812 03:14:09.168001 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 2681
I20260812 03:14:09.183092 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 2907
I20260812 03:14:09.194216 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 2302
2026-08-12T03:14:09Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (888 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:14:09 dist-test-slave-7nm7 krb5kdc[3155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:14:09 dist-test-slave-7nm7 krb5kdc[3155](info): set up 2 sockets
Aug 12 03:14:09 dist-test-slave-7nm7 krb5kdc[3155](info): commencing operation
krb5kdc: starting...
W20260812 03:14:10.517439 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:14:10 dist-test-slave-7nm7 krb5kdc[3155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:14:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:14:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:14:10.657799 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:46741
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:34357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:46741
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:14:10.776237 3179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:14:10.776520 3179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:14:10.776564 3179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:14:10.780769 3179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:14:10.780833 3179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:14:10.780853 3179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:14:10.780870 3179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:14:10.780885 3179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:14:10.786967 3179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:34357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:46741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:46741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:14:10.788167 3179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:14:10.789248 3179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:14:10.797936 3192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:10.797953 3190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:14:10.798193 3179 server_base.cc:1034] running on GCE node
W20260812 03:14:10.797941 3189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:14:10.798923 3179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:14:10.799947 3179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:14:10.801242 3179 hybrid_clock.cc:648] HybridClock initialized: now 1786504450801224 us; error 35 us; skew 500 ppm
Aug 12 03:14:10 dist-test-slave-7nm7 krb5kdc[3155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504450, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:14:10.804168 3179 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:14:10.805182 3179 webserver.cc:460] Webserver started at http://127.16.217.126:44923/ using document root <none> and password file <none>
I20260812 03:14:10.805747 3179 fs_manager.cc:359] Metadata directory not provided
I20260812 03:14:10.805809 3179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:14:10.805979 3179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:14:10.808100 3179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "308c964c6992464b87ced2532ff2e8fb"
format_stamp: "Formatted at 2026-08-12 03:14:10 on dist-test-slave-7nm7"
server_key: "492b7448d571178f591249e3788cd48a"
server_key_iv: "363c3dfa3e9430910a6870853073aa47"
server_key_version: "encryptionkey@0"
I20260812 03:14:10.808647 3179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "308c964c6992464b87ced2532ff2e8fb"
format_stamp: "Formatted at 2026-08-12 03:14:10 on dist-test-slave-7nm7"
server_key: "492b7448d571178f591249e3788cd48a"
server_key_iv: "363c3dfa3e9430910a6870853073aa47"
server_key_version: "encryptionkey@0"
I20260812 03:14:10.812472 3179 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:14:10.815043 3204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:10.816390 3179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:14:10.816502 3179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "308c964c6992464b87ced2532ff2e8fb"
format_stamp: "Formatted at 2026-08-12 03:14:10 on dist-test-slave-7nm7"
server_key: "492b7448d571178f591249e3788cd48a"
server_key_iv: "363c3dfa3e9430910a6870853073aa47"
server_key_version: "encryptionkey@0"
I20260812 03:14:10.816598 3179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:14:10.832832 3179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:14:10.836323 3179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:14:10.836498 3179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:14:10.844522 3179 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:46741
I20260812 03:14:10.844609 3296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:46741 every 8 connection(s)
I20260812 03:14:10.845646 3179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:14:10.846087 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 3179
I20260812 03:14:10.846261 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:14:10.846566 17253 external_mini_cluster.cc:1428] Setting key 63015e62ff5b3da5733863c952a6fea0
I20260812 03:14:10.848997 3297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:14:10.855886 3297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb: Bootstrap starting.
Aug 12 03:14:10 dist-test-slave-7nm7 krb5kdc[3155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:14:10.858570 3297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb: Neither blocks nor log segments found. Creating new log.
I20260812 03:14:10.859351 3297 log.cc:824] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb: Log is configured to *not* fsync() on all Append() calls
I20260812 03:14:10.861927 3297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb: No bootstrap required, opened a new log
I20260812 03:14:10.862195 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:10.847914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41110 (local address 127.16.217.126:46741)
0812 03:14:10.848543 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:10.848567 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:10.848618 (+ 51us) server_negotiation.cc:407] Connection header received
0812 03:14:10.849381 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:10.849401 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:10.849743 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:10.850025 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:10.851032 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:10.852069 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:10.853198 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:10.853681 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:10.853800 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:10.856803 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:14:10.856809 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:14:10.856823 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:14:10.856854 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:14:10.858850 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:14:10.859324 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:14:10.859338 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:14:10.859343 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:14:10.859460 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:14:10.859863 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:14:10.859870 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:14:10.859871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:14:10.860446 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:14:10.860599 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:10.861025 (+ 426us) server_negotiation.cc:299] Negotiation successful
0812 03:14:10.861407 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":164,"threads_started":1}
I20260812 03:14:10.865120 3297 raft_consensus.cc:348] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "308c964c6992464b87ced2532ff2e8fb" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46741 } }
I20260812 03:14:10.865377 3297 raft_consensus.cc:374] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:14:10.865463 3297 raft_consensus.cc:729] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 308c964c6992464b87ced2532ff2e8fb, State: Initialized, Role: FOLLOWER
I20260812 03:14:10.865952 3297 consensus_queue.cc:260] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "308c964c6992464b87ced2532ff2e8fb" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46741 } }
I20260812 03:14:10.866130 3297 raft_consensus.cc:388] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:14:10.866230 3297 raft_consensus.cc:482] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:14:10.866361 3297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:14:10.867333 3297 raft_consensus.cc:504] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "308c964c6992464b87ced2532ff2e8fb" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46741 } }
I20260812 03:14:10.867733 3297 leader_election.cc:302] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 308c964c6992464b87ced2532ff2e8fb; no voters:
I20260812 03:14:10.868091 3297 leader_election.cc:288] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:14:10.868494 3303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:14:10.868695 3303 raft_consensus.cc:686] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb [term 1 LEADER]: Becoming Leader. State: Replica: 308c964c6992464b87ced2532ff2e8fb, State: Running, Role: LEADER
I20260812 03:14:10.868986 3303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "308c964c6992464b87ced2532ff2e8fb" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46741 } }
I20260812 03:14:10.869457 3297 sys_catalog.cc:564] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:14:10.870708 3306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "308c964c6992464b87ced2532ff2e8fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "308c964c6992464b87ced2532ff2e8fb" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46741 } } }
I20260812 03:14:10.870844 3306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb [sys.catalog]: This master's current role is: LEADER
I20260812 03:14:10.871121 3302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 308c964c6992464b87ced2532ff2e8fb. Latest consensus state: current_term: 1 leader_uuid: "308c964c6992464b87ced2532ff2e8fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "308c964c6992464b87ced2532ff2e8fb" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46741 } } }
I20260812 03:14:10.871222 3302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb [sys.catalog]: This master's current role is: LEADER
I20260812 03:14:10.871268 3317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:14:10.874562 3317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:14:10.879842 3317 catalog_manager.cc:1347] Generated new cluster ID: 3d2e57b5c941499b96c9dc9082420a88
I20260812 03:14:10.879911 3317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:14:10.896471 3317 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:14:10.897779 3317 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:14:10.903872 3317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb: Generated new TSK 0
I20260812 03:14:10.904578 3317 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
I20260812 03:14:10.936841 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:46741
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:34357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:14:11.056002 3344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:14:11.056334 3344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:14:11.056421 3344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:14:11.060949 3344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:14:11.061061 3344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:14:11.061162 3344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:14:11.067040 3344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:34357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:46741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:14:11.068253 3344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:14:11.069357 3344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:14:11.080056 3355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:11.080070 3354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:11.080057 3357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:14:11.080422 3344 server_base.cc:1034] running on GCE node
I20260812 03:14:11.080940 3344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:14:11.081648 3344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:14:11.082836 3344 hybrid_clock.cc:648] HybridClock initialized: now 1786504451082817 us; error 38 us; skew 500 ppm
Aug 12 03:14:11 dist-test-slave-7nm7 krb5kdc[3155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504451, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:14:11.085783 3344 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:14:11.086946 3344 webserver.cc:460] Webserver started at http://127.16.217.65:38615/ using document root <none> and password file <none>
I20260812 03:14:11.087549 3344 fs_manager.cc:359] Metadata directory not provided
I20260812 03:14:11.087658 3344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:14:11.087951 3344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:14:11.090283 3344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "e840f035c4304a71b553987947cb889c"
format_stamp: "Formatted at 2026-08-12 03:14:11 on dist-test-slave-7nm7"
server_key: "06536a8b8b9b22531a869ef34de9dff9"
server_key_iv: "0437dcf8bfbad88f99fb367a5d7736fe"
server_key_version: "encryptionkey@0"
I20260812 03:14:11.090853 3344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "e840f035c4304a71b553987947cb889c"
format_stamp: "Formatted at 2026-08-12 03:14:11 on dist-test-slave-7nm7"
server_key: "06536a8b8b9b22531a869ef34de9dff9"
server_key_iv: "0437dcf8bfbad88f99fb367a5d7736fe"
server_key_version: "encryptionkey@0"
I20260812 03:14:11.095499 3344 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:14:11.098098 3370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:11.099540 3344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:14:11.099650 3344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "e840f035c4304a71b553987947cb889c"
format_stamp: "Formatted at 2026-08-12 03:14:11 on dist-test-slave-7nm7"
server_key: "06536a8b8b9b22531a869ef34de9dff9"
server_key_iv: "0437dcf8bfbad88f99fb367a5d7736fe"
server_key_version: "encryptionkey@0"
I20260812 03:14:11.099745 3344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:14:11.123031 3344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:14:11.126606 3344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:14:11.126777 3344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:14:11.127349 3344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:14:11.128189 3344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:14:11.128239 3344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:11.128281 3344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:14:11.128296 3344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:11.137989 3344 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:36187
I20260812 03:14:11.138017 3580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:36187 every 8 connection(s)
I20260812 03:14:11.138980 3344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:14:11.144886 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 3344
I20260812 03:14:11.145031 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:14:11.145323 17253 external_mini_cluster.cc:1428] Setting key 2c7940a1a1b1087930acb4d967c3f5d3
Aug 12 03:14:11 dist-test-slave-7nm7 krb5kdc[3155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504451, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:14:11.154904 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.140883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:52261 (local address 127.16.217.126:46741)
0812 03:14:11.141063 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:11.141068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:11.141917 (+ 849us) server_negotiation.cc:407] Connection header received
0812 03:14:11.143334 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:11.143344 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:11.143427 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:11.143505 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:11.144731 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.145404 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.146724 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.147051 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.147176 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:11.150078 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:14:11.150082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:14:11.150084 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:14:11.150115 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:14:11.152089 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:14:11.152658 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:14:11.152664 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:14:11.152669 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:14:11.152770 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:14:11.153358 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:14:11.153365 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:14:11.153370 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:14:11.153732 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:14:11.153872 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:11.154509 (+ 637us) server_negotiation.cc:299] Negotiation successful
0812 03:14:11.154687 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 03:14:11.155692 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.141285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:46741 (local address 127.16.217.65:52261)
0812 03:14:11.141756 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:11.141800 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:11.143103 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:11.143678 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:11.143685 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:11.144060 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:11.144538 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:11.144551 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.145540 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.145544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:11.146581 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:11.146591 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.147231 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.147234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:11.147380 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:11.148029 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:11.148046 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:11.149892 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:14:11.152231 (+ 2339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:14:11.152237 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:14:11.152252 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:14:11.152546 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:14:11.152901 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:14:11.152904 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:14:11.152905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:14:11.153162 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:14:11.153874 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:14:11.153881 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:14:11.154170 (+ 289us) client_negotiation.cc:769] Sending connection context
0812 03:14:11.154421 (+ 251us) client_negotiation.cc:240] Negotiation successful
0812 03:14:11.154716 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":114,"threads_started":1}
I20260812 03:14:11.157099 3581 heartbeater.cc:347] Connected to a master server at 127.16.217.126:46741
WARNING: no policy specified for kudu/127.16.217.66@KRBTEST.COM; defaulting to no policy
I20260812 03:14:11.157393 3581 heartbeater.cc:464] Registering TS with master...
I20260812 03:14:11.157929 3581 heartbeater.cc:511] Master 127.16.217.126:46741 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.66@KRBTEST.COM" created.
I20260812 03:14:11.159759 3230 ts_manager.cc:194] Registered new tserver with Master: e840f035c4304a71b553987947cb889c (127.16.217.65:36187)
I20260812 03:14:11.160933 3230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:52261
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
I20260812 03:14:11.177007 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:46741
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:34357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:14:11.293164 3588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:14:11.293428 3588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:14:11.293520 3588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:14:11.297546 3588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:14:11.297605 3588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:14:11.297701 3588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:14:11.303036 3588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:34357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:46741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:14:11.304177 3588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:14:11.305241 3588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:14:11.315039 3599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:11.315057 3602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:11.315129 3597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:14:11.315223 3588 server_base.cc:1034] running on GCE node
I20260812 03:14:11.315673 3588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:14:11.316334 3588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:14:11.317538 3588 hybrid_clock.cc:648] HybridClock initialized: now 1786504451317509 us; error 36 us; skew 500 ppm
Aug 12 03:14:11 dist-test-slave-7nm7 krb5kdc[3155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504451, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:14:11.320508 3588 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:14:11.321403 3588 webserver.cc:460] Webserver started at http://127.16.217.66:34271/ using document root <none> and password file <none>
I20260812 03:14:11.321931 3588 fs_manager.cc:359] Metadata directory not provided
I20260812 03:14:11.321993 3588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:14:11.322161 3588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:14:11.324234 3588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "898be7030e914b399fe5f63b792376d2"
format_stamp: "Formatted at 2026-08-12 03:14:11 on dist-test-slave-7nm7"
server_key: "20d1552d74438c265a62a78fa93345e6"
server_key_iv: "badec2ded18b704e0b08fa7dc4467396"
server_key_version: "encryptionkey@0"
I20260812 03:14:11.324867 3588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "898be7030e914b399fe5f63b792376d2"
format_stamp: "Formatted at 2026-08-12 03:14:11 on dist-test-slave-7nm7"
server_key: "20d1552d74438c265a62a78fa93345e6"
server_key_iv: "badec2ded18b704e0b08fa7dc4467396"
server_key_version: "encryptionkey@0"
I20260812 03:14:11.328670 3588 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 03:14:11.332192 3615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:11.333401 3588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:14:11.333536 3588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "898be7030e914b399fe5f63b792376d2"
format_stamp: "Formatted at 2026-08-12 03:14:11 on dist-test-slave-7nm7"
server_key: "20d1552d74438c265a62a78fa93345e6"
server_key_iv: "badec2ded18b704e0b08fa7dc4467396"
server_key_version: "encryptionkey@0"
I20260812 03:14:11.333652 3588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:14:11.348181 3588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:14:11.351212 3588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:14:11.351411 3588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:14:11.352001 3588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:14:11.352833 3588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:14:11.352880 3588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:11.352917 3588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:14:11.352978 3588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:11.362026 3588 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:36025
I20260812 03:14:11.362099 3836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:36025 every 8 connection(s)
I20260812 03:14:11.362982 3588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:14:11.364161 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 3588
I20260812 03:14:11.364284 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:14:11.364545 17253 external_mini_cluster.cc:1428] Setting key 0afb7f075e69a60c70488da583196fcc
Aug 12 03:14:11 dist-test-slave-7nm7 krb5kdc[3155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504451, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.67@KRBTEST.COM" created.
I20260812 03:14:11.380254 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.364842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:56237 (local address 127.16.217.126:46741)
0812 03:14:11.365032 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:11.365035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:11.366090 (+ 1055us) server_negotiation.cc:407] Connection header received
0812 03:14:11.368205 (+ 2115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:11.368208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:11.368252 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:11.368325 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:11.369673 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.370544 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.372397 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.372713 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.372803 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:11.375907 (+ 3104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:14:11.375911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:14:11.375914 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:14:11.375939 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:14:11.377445 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:14:11.378008 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:14:11.378011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:14:11.378012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:14:11.378137 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:14:11.378693 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:14:11.378699 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:14:11.378702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:14:11.379032 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:14:11.379170 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:11.379856 (+ 686us) server_negotiation.cc:299] Negotiation successful
0812 03:14:11.380060 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260812 03:14:11.381417 3843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.365312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:46741 (local address 127.16.217.66:56237)
0812 03:14:11.365911 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:11.365968 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:11.367951 (+ 1983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:11.368563 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:11.368573 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:11.368936 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:11.369483 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:11.369499 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.370715 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.370721 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:11.372241 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:11.372257 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.372886 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.372890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:11.373110 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:11.373809 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:11.373827 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:11.375695 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:14:11.377567 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:14:11.377572 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:14:11.377586 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:14:11.377911 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:14:11.378298 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:14:11.378301 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:14:11.378303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:14:11.378531 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:14:11.379172 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:14:11.379180 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:14:11.379512 (+ 332us) client_negotiation.cc:769] Sending connection context
0812 03:14:11.379736 (+ 224us) client_negotiation.cc:240] Negotiation successful
0812 03:14:11.380210 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":144,"threads_started":1}
I20260812 03:14:11.382870 3838 heartbeater.cc:347] Connected to a master server at 127.16.217.126:46741
I20260812 03:14:11.383143 3838 heartbeater.cc:464] Registering TS with master...
I20260812 03:14:11.383671 3838 heartbeater.cc:511] Master 127.16.217.126:46741 requested a full tablet report, sending...
I20260812 03:14:11.384848 3230 ts_manager.cc:194] Registered new tserver with Master: 898be7030e914b399fe5f63b792376d2 (127.16.217.66:36025)
I20260812 03:14:11.385447 3230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:56237
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
I20260812 03:14:11.397883 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:46741
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:34357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:14:11.515676 3850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:14:11.516026 3850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:14:11.516108 3850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:14:11.520651 3850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:14:11.520750 3850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:14:11.520844 3850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:14:11.526767 3850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:34357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:46741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:14:11.527983 3850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:14:11.529080 3850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:14:11.538976 3858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:11.538995 3861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:14:11.539016 3863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:14:11.539397 3850 server_base.cc:1034] running on GCE node
I20260812 03:14:11.539768 3850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:14:11.540424 3850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:14:11.541594 3850 hybrid_clock.cc:648] HybridClock initialized: now 1786504451541579 us; error 25 us; skew 500 ppm
Aug 12 03:14:11 dist-test-slave-7nm7 krb5kdc[3155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504451, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:14:11.549392 3850 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:14:11.550535 3850 webserver.cc:460] Webserver started at http://127.16.217.67:39703/ using document root <none> and password file <none>
I20260812 03:14:11.551180 3850 fs_manager.cc:359] Metadata directory not provided
I20260812 03:14:11.551246 3850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:14:11.551471 3850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:14:11.553618 3850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "24352d182e984c069bcb5eee8e3d5278"
format_stamp: "Formatted at 2026-08-12 03:14:11 on dist-test-slave-7nm7"
server_key: "e99aa7f4277098c3ef37f9053fccfa64"
server_key_iv: "879694f4b3548c58ccdd3e61fcaf97e2"
server_key_version: "encryptionkey@0"
I20260812 03:14:11.554196 3850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "24352d182e984c069bcb5eee8e3d5278"
format_stamp: "Formatted at 2026-08-12 03:14:11 on dist-test-slave-7nm7"
server_key: "e99aa7f4277098c3ef37f9053fccfa64"
server_key_iv: "879694f4b3548c58ccdd3e61fcaf97e2"
server_key_version: "encryptionkey@0"
I20260812 03:14:11.558408 3850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:14:11.561024 3874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:11.562527 3850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:14:11.562665 3850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "24352d182e984c069bcb5eee8e3d5278"
format_stamp: "Formatted at 2026-08-12 03:14:11 on dist-test-slave-7nm7"
server_key: "e99aa7f4277098c3ef37f9053fccfa64"
server_key_iv: "879694f4b3548c58ccdd3e61fcaf97e2"
server_key_version: "encryptionkey@0"
I20260812 03:14:11.562788 3850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:14:11.600564 3850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:14:11.604374 3850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:14:11.604614 3850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:14:11.605257 3850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:14:11.606200 3850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:14:11.606302 3850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:11.606370 3850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:14:11.606411 3850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:14:11.615495 3850 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:37289
I20260812 03:14:11.615518 4077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:37289 every 8 connection(s)
I20260812 03:14:11.616454 3850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:14:11 dist-test-slave-7nm7 krb5kdc[3155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504451, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:14:11.626559 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 3850
I20260812 03:14:11.626713 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:14:11.627061 17253 external_mini_cluster.cc:1428] Setting key c3b08dde0d5ab2e9c51dd32f15e6d04e
I20260812 03:14:11.630507 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.618356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:39695 (local address 127.16.217.126:46741)
0812 03:14:11.618511 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:11.618516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:11.619423 (+ 907us) server_negotiation.cc:407] Connection header received
0812 03:14:11.620926 (+ 1503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:11.620929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:11.620979 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:11.621054 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:11.622325 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.622861 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.623983 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.624207 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.624300 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:11.626851 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:14:11.626855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:14:11.626857 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:14:11.626881 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:14:11.628386 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:14:11.628934 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:14:11.628938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:14:11.628939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:14:11.629015 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:14:11.629373 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:14:11.629376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:14:11.629378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:14:11.629573 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:14:11.629680 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:11.630189 (+ 509us) server_negotiation.cc:299] Negotiation successful
0812 03:14:11.630343 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:14:11.631387 4081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.618775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:46741 (local address 127.16.217.67:39695)
0812 03:14:11.619272 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:11.619316 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:11.620669 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:11.621218 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:11.621224 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:11.621570 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:14:11.622114 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:11.622128 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.622980 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.622982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:11.623850 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:11.623860 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.624353 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.624356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:11.624500 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:11.625101 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:14:11.625118 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:14:11.626641 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:14:11.628514 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:14:11.628522 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:14:11.628537 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:14:11.628810 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:14:11.629111 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:14:11.629115 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:14:11.629117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:14:11.629265 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:14:11.629681 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:14:11.629689 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:14:11.629944 (+ 255us) client_negotiation.cc:769] Sending connection context
0812 03:14:11.630150 (+ 206us) client_negotiation.cc:240] Negotiation successful
0812 03:14:11.630511 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":122,"threads_started":1}
I20260812 03:14:11.632591 4078 heartbeater.cc:347] Connected to a master server at 127.16.217.126:46741
I20260812 03:14:11.632822 4078 heartbeater.cc:464] Registering TS with master...
I20260812 03:14:11.633307 4078 heartbeater.cc:511] Master 127.16.217.126:46741 requested a full tablet report, sending...
I20260812 03:14:11.634405 3230 ts_manager.cc:194] Registered new tserver with Master: 24352d182e984c069bcb5eee8e3d5278 (127.16.217.67:37289)
I20260812 03:14:11.634987 3230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:39695
I20260812 03:14:11.641041 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:14:11.650367 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.643047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41122 (local address 127.16.217.126:46741)
0812 03:14:11.643184 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:11.643187 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:11.643346 (+ 159us) server_negotiation.cc:407] Connection header received
0812 03:14:11.643446 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:11.643448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:11.643487 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:11.643556 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:14:11.644207 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.644828 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.645668 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.645869 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.645945 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:11.647259 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:14:11.647262 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:14:11.647264 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:14:11.647287 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:14:11.648715 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:14:11.649099 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:14:11.649101 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:14:11.649102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:14:11.649178 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:14:11.649491 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:14:11.649494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:14:11.649495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:14:11.649718 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:14:11.649832 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:11.650049 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 03:14:11.650156 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:14:11.653079 3230 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:14:11.657164 3230 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:14:11.675467 4095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.669533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48878 (local address 127.16.217.65:36187)
0812 03:14:11.669848 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:11.669855 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:11.670215 (+ 360us) server_negotiation.cc:407] Connection header received
0812 03:14:11.670922 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:11.670928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:11.671130 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:11.671302 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:14:11.671600 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.672477 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.674150 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.674770 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.674901 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:11.675059 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:11.675211 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 03:14:11.675306 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":125,"threads_started":1}
I20260812 03:14:11.675450 4094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.669361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:36187 (local address 127.0.0.1:48878)
0812 03:14:11.670008 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:11.670075 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:11.670741 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:11.671275 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:11.671282 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:11.671306 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:14:11.671465 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:11.671473 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.672597 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.672601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:11.673988 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:11.674000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.674901 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.674905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:11.675021 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:11.675059 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 03:14:11.675178 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 03:14:11.675303 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"spinlock_wait_cycles":840192,"thread_start_us":70,"threads_started":1}
I20260812 03:14:11.675836 4099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.670299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:37289 (local address 127.0.0.1:48594)
0812 03:14:11.670524 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:11.670537 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:11.670738 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:11.671829 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:11.671833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:11.671850 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:14:11.671936 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:11.671942 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.673040 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.673043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:11.674144 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:11.674153 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.675426 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.675429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:11.675521 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:11.675545 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:14:11.675629 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 03:14:11.675693 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":3,"spinlock_wait_cycles":161280,"thread_start_us":98,"threads_started":1}
I20260812 03:14:11.676400 4100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.670428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48594 (local address 127.16.217.67:37289)
0812 03:14:11.670717 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:11.670730 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:11.670754 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:14:11.671492 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:11.671500 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:11.671688 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:11.671850 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:14:11.672031 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.672883 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.674290 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.675094 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.675221 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:11.675363 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:11.676142 (+ 779us) server_negotiation.cc:299] Negotiation successful
0812 03:14:11.676243 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":103,"threads_started":1}
I20260812 03:14:11.676852 4098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.669361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:36025 (local address 127.0.0.1:39230)
0812 03:14:11.670191 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:11.670204 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:11.670924 (+ 720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:11.672604 (+ 1680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:11.672606 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:11.672624 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:14:11.672713 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:11.672717 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.673951 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.673955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:11.675118 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:11.675130 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.676184 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.676187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:11.676284 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:11.676309 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:14:11.676373 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:14:11.676437 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"mutex_wait_us":532,"spinlock_wait_cycles":1280,"thread_start_us":100,"threads_started":1}
I20260812 03:14:11.676893 4097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.669611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39230 (local address 127.16.217.66:36025)
0812 03:14:11.671180 (+ 1569us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:11.671188 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:11.671207 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:14:11.671292 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:11.671298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:11.671461 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:11.671622 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:14:11.672830 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.673810 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.675252 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.676054 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.676174 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:11.676579 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:11.676633 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 03:14:11.676722 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1474,"thread_start_us":146,"threads_started":1}
I20260812 03:14:11.679116 3460 tablet_service.cc:1467] Processing CreateTablet for tablet eacf6dead0794be2ba611936cb51a5e6 (DEFAULT_TABLE table=test-table [id=0a62bdd87e894c5192e49e5e5feaca2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:14:11.679620 3960 tablet_service.cc:1467] Processing CreateTablet for tablet eacf6dead0794be2ba611936cb51a5e6 (DEFAULT_TABLE table=test-table [id=0a62bdd87e894c5192e49e5e5feaca2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:14:11.679616 3721 tablet_service.cc:1467] Processing CreateTablet for tablet eacf6dead0794be2ba611936cb51a5e6 (DEFAULT_TABLE table=test-table [id=0a62bdd87e894c5192e49e5e5feaca2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:14:11.680143 3460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eacf6dead0794be2ba611936cb51a5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:14:11.680586 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eacf6dead0794be2ba611936cb51a5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:14:11.680605 3960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eacf6dead0794be2ba611936cb51a5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:14:11.687260 4105 tablet_bootstrap.cc:492] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c: Bootstrap starting.
I20260812 03:14:11.688591 4108 tablet_bootstrap.cc:492] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278: Bootstrap starting.
I20260812 03:14:11.688680 4107 tablet_bootstrap.cc:492] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2: Bootstrap starting.
I20260812 03:14:11.689754 4105 tablet_bootstrap.cc:654] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c: Neither blocks nor log segments found. Creating new log.
I20260812 03:14:11.690407 4108 tablet_bootstrap.cc:654] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278: Neither blocks nor log segments found. Creating new log.
I20260812 03:14:11.690764 4105 log.cc:824] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:14:11.691264 4107 tablet_bootstrap.cc:654] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2: Neither blocks nor log segments found. Creating new log.
I20260812 03:14:11.691641 4108 log.cc:824] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278: Log is configured to *not* fsync() on all Append() calls
I20260812 03:14:11.691926 4107 log.cc:824] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2: Log is configured to *not* fsync() on all Append() calls
I20260812 03:14:11.693713 4108 tablet_bootstrap.cc:492] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278: No bootstrap required, opened a new log
I20260812 03:14:11.693753 4107 tablet_bootstrap.cc:492] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2: No bootstrap required, opened a new log
I20260812 03:14:11.693943 4108 ts_tablet_manager.cc:1397] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 03:14:11.693950 4107 ts_tablet_manager.cc:1397] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:14:11.695644 4105 tablet_bootstrap.cc:492] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c: No bootstrap required, opened a new log
I20260812 03:14:11.695839 4105 ts_tablet_manager.cc:1397] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260812 03:14:11.697113 4108 raft_consensus.cc:348] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e840f035c4304a71b553987947cb889c" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36187 } } peers { permanent_uuid: "898be7030e914b399fe5f63b792376d2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36025 } } peers { permanent_uuid: "24352d182e984c069bcb5eee8e3d5278" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37289 } }
I20260812 03:14:11.697363 4108 raft_consensus.cc:374] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:14:11.697429 4108 raft_consensus.cc:729] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24352d182e984c069bcb5eee8e3d5278, State: Initialized, Role: FOLLOWER
I20260812 03:14:11.697984 4108 consensus_queue.cc:260] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e840f035c4304a71b553987947cb889c" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36187 } } peers { permanent_uuid: "898be7030e914b399fe5f63b792376d2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36025 } } peers { permanent_uuid: "24352d182e984c069bcb5eee8e3d5278" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37289 } }
I20260812 03:14:11.698005 4107 raft_consensus.cc:348] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e840f035c4304a71b553987947cb889c" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36187 } } peers { permanent_uuid: "898be7030e914b399fe5f63b792376d2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36025 } } peers { permanent_uuid: "24352d182e984c069bcb5eee8e3d5278" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37289 } }
I20260812 03:14:11.698277 4107 raft_consensus.cc:374] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:14:11.698328 4107 raft_consensus.cc:729] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 898be7030e914b399fe5f63b792376d2, State: Initialized, Role: FOLLOWER
I20260812 03:14:11.698778 4107 consensus_queue.cc:260] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e840f035c4304a71b553987947cb889c" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36187 } } peers { permanent_uuid: "898be7030e914b399fe5f63b792376d2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36025 } } peers { permanent_uuid: "24352d182e984c069bcb5eee8e3d5278" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37289 } }
I20260812 03:14:11.699110 4108 ts_tablet_manager.cc:1428] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260812 03:14:11.699492 4078 heartbeater.cc:503] Master 127.16.217.126:46741 was elected leader, sending a full tablet report...
I20260812 03:14:11.699824 4107 ts_tablet_manager.cc:1428] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260812 03:14:11.700388 4105 raft_consensus.cc:348] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e840f035c4304a71b553987947cb889c" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36187 } } peers { permanent_uuid: "898be7030e914b399fe5f63b792376d2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36025 } } peers { permanent_uuid: "24352d182e984c069bcb5eee8e3d5278" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37289 } }
I20260812 03:14:11.700614 3838 heartbeater.cc:503] Master 127.16.217.126:46741 was elected leader, sending a full tablet report...
I20260812 03:14:11.700675 4105 raft_consensus.cc:374] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:14:11.700734 4105 raft_consensus.cc:729] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e840f035c4304a71b553987947cb889c, State: Initialized, Role: FOLLOWER
I20260812 03:14:11.701232 4105 consensus_queue.cc:260] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e840f035c4304a71b553987947cb889c" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36187 } } peers { permanent_uuid: "898be7030e914b399fe5f63b792376d2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36025 } } peers { permanent_uuid: "24352d182e984c069bcb5eee8e3d5278" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37289 } }
I20260812 03:14:11.702261 4105 ts_tablet_manager.cc:1428] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260812 03:14:11.702585 3581 heartbeater.cc:503] Master 127.16.217.126:46741 was elected leader, sending a full tablet report...
I20260812 03:14:11.732695 4118 raft_consensus.cc:482] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:14:11.732920 4118 raft_consensus.cc:504] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e840f035c4304a71b553987947cb889c" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36187 } } peers { permanent_uuid: "898be7030e914b399fe5f63b792376d2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36025 } } peers { permanent_uuid: "24352d182e984c069bcb5eee8e3d5278" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37289 } }
I20260812 03:14:11.734131 4118 leader_election.cc:288] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 898be7030e914b399fe5f63b792376d2 (127.16.217.66:36025), 24352d182e984c069bcb5eee8e3d5278 (127.16.217.67:37289)
I20260812 03:14:11.738866 4097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.734624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:38681 (local address 127.16.217.66:36025)
0812 03:14:11.734787 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:11.734791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:11.734928 (+ 137us) server_negotiation.cc:407] Connection header received
0812 03:14:11.735179 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:11.735186 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:11.735248 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:11.735304 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:14:11.735816 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.736440 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.737787 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.738285 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.738402 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:11.738467 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:11.738650 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 03:14:11.738736 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 03:14:11.738897 4100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.735132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:36487 (local address 127.16.217.67:37289)
0812 03:14:11.735287 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:11.735293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:11.735465 (+ 172us) server_negotiation.cc:407] Connection header received
0812 03:14:11.735617 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:11.735622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:11.735684 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:11.735760 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:14:11.736077 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.736601 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.737852 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.738316 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.738448 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:11.738504 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:11.738716 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 03:14:11.738787 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:14:11.738886 4119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.734508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:36025 (local address 127.16.217.65:38681)
0812 03:14:11.734834 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:11.734852 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:11.735017 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:11.735480 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:11.735485 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:11.735509 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:14:11.735661 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:11.735668 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.736561 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.736565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:11.737661 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:11.737671 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.738403 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.738410 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:11.738506 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:11.738538 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 03:14:11.738611 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:14:11.738702 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":36,"thread_start_us":137,"threads_started":1}
I20260812 03:14:11.738899 4120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.734963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:37289 (local address 127.16.217.65:36487)
0812 03:14:11.735356 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:11.735371 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:14:11.735501 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:14:11.735818 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:14:11.735821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:14:11.735841 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:14:11.735956 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:11.735962 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.736738 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.736741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:11.737661 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:14:11.737671 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.738458 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.738462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:14:11.738563 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:11.738582 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 03:14:11.738680 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 03:14:11.738743 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":41,"thread_start_us":89,"threads_started":1}
I20260812 03:14:11.739656 3755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eacf6dead0794be2ba611936cb51a5e6" candidate_uuid: "e840f035c4304a71b553987947cb889c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "898be7030e914b399fe5f63b792376d2" is_pre_election: true
I20260812 03:14:11.739673 3993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eacf6dead0794be2ba611936cb51a5e6" candidate_uuid: "e840f035c4304a71b553987947cb889c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24352d182e984c069bcb5eee8e3d5278" is_pre_election: true
I20260812 03:14:11.739957 3993 raft_consensus.cc:2454] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e840f035c4304a71b553987947cb889c in term 0.
I20260812 03:14:11.739964 3755 raft_consensus.cc:2454] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e840f035c4304a71b553987947cb889c in term 0.
I20260812 03:14:11.740428 3374 leader_election.cc:302] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 898be7030e914b399fe5f63b792376d2, e840f035c4304a71b553987947cb889c; no voters:
I20260812 03:14:11.740675 4118 raft_consensus.cc:2781] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:14:11.740767 4118 raft_consensus.cc:482] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:14:11.740820 4118 raft_consensus.cc:3037] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:14:11.741987 4118 raft_consensus.cc:504] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e840f035c4304a71b553987947cb889c" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36187 } } peers { permanent_uuid: "898be7030e914b399fe5f63b792376d2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36025 } } peers { permanent_uuid: "24352d182e984c069bcb5eee8e3d5278" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37289 } }
I20260812 03:14:11.742352 4118 leader_election.cc:288] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [CANDIDATE]: Term 1 election: Requested vote from peers 898be7030e914b399fe5f63b792376d2 (127.16.217.66:36025), 24352d182e984c069bcb5eee8e3d5278 (127.16.217.67:37289)
I20260812 03:14:11.742735 3755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eacf6dead0794be2ba611936cb51a5e6" candidate_uuid: "e840f035c4304a71b553987947cb889c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "898be7030e914b399fe5f63b792376d2"
I20260812 03:14:11.742750 3993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eacf6dead0794be2ba611936cb51a5e6" candidate_uuid: "e840f035c4304a71b553987947cb889c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24352d182e984c069bcb5eee8e3d5278"
I20260812 03:14:11.742854 3755 raft_consensus.cc:3037] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:14:11.742883 3993 raft_consensus.cc:3037] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:14:11.743904 3755 raft_consensus.cc:2454] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e840f035c4304a71b553987947cb889c in term 1.
I20260812 03:14:11.743981 3993 raft_consensus.cc:2454] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e840f035c4304a71b553987947cb889c in term 1.
I20260812 03:14:11.744187 3374 leader_election.cc:302] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 898be7030e914b399fe5f63b792376d2, e840f035c4304a71b553987947cb889c; no voters:
I20260812 03:14:11.744375 4118 raft_consensus.cc:2781] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:14:11.744619 4118 raft_consensus.cc:686] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [term 1 LEADER]: Becoming Leader. State: Replica: e840f035c4304a71b553987947cb889c, State: Running, Role: LEADER
I20260812 03:14:11.744992 4118 consensus_queue.cc:237] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e840f035c4304a71b553987947cb889c" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36187 } } peers { permanent_uuid: "898be7030e914b399fe5f63b792376d2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36025 } } peers { permanent_uuid: "24352d182e984c069bcb5eee8e3d5278" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37289 } }
I20260812 03:14:11.748391 3227 catalog_manager.cc:5495] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c reported cstate change: term changed from 0 to 1, leader changed from <none> to e840f035c4304a71b553987947cb889c (127.16.217.65). New cstate: current_term: 1 leader_uuid: "e840f035c4304a71b553987947cb889c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e840f035c4304a71b553987947cb889c" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 36187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "898be7030e914b399fe5f63b792376d2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24352d182e984c069bcb5eee8e3d5278" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37289 } health_report { overall_health: UNKNOWN } } }
I20260812 03:14:11.775730 4095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.771869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48888 (local address 127.16.217.65:36187)
0812 03:14:11.772216 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0812 03:14:11.772220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:14:11.772234 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:14:11.772313 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:14:11.772315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:14:11.772372 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:14:11.772514 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:14:11.772784 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.773314 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.774159 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:14:11.774461 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:14:11.774578 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:14:11.774891 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:14:11.775325 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:14:11.775461 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 03:14:11.775521 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 03:14:11.775578 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266}
I20260812 03:14:11.781638 3755 raft_consensus.cc:1261] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2 [term 1 FOLLOWER]: Refusing update from remote peer e840f035c4304a71b553987947cb889c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:14:11.781653 3993 raft_consensus.cc:1261] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278 [term 1 FOLLOWER]: Refusing update from remote peer e840f035c4304a71b553987947cb889c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:14:11.782507 4121 consensus_queue.cc:1035] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24352d182e984c069bcb5eee8e3d5278" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 37289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:14:11.782743 4118 consensus_queue.cc:1035] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [LEADER]: Connected to new peer: Peer: permanent_uuid: "898be7030e914b399fe5f63b792376d2" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 36025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:14:11.790618 4129 mvcc.cc:205] Tried to move back new op lower bound from 7317522234490175488 to 7317522234350718976. Current Snapshot: MvccSnapshot[applied={T|T < 7317522234490175488}]
I20260812 03:14:11.794806 4130 mvcc.cc:205] Tried to move back new op lower bound from 7317522234490175488 to 7317522234350718976. Current Snapshot: MvccSnapshot[applied={T|T < 7317522234490175488}]
I20260812 03:14:11.798425 3227 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:14:11.798620 3227 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:14:11.801030 3227 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 308c964c6992464b87ced2532ff2e8fb: Sending DeleteTablet for 3 replicas of tablet eacf6dead0794be2ba611936cb51a5e6
I20260812 03:14:11.801873 3460 tablet_service.cc:1514] Processing DeleteTablet for tablet eacf6dead0794be2ba611936cb51a5e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:14:11 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:48878
I20260812 03:14:11.802047 3960 tablet_service.cc:1514] Processing DeleteTablet for tablet eacf6dead0794be2ba611936cb51a5e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:14:11 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:48594
I20260812 03:14:11.802060 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet eacf6dead0794be2ba611936cb51a5e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:14:11 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:39230
I20260812 03:14:11.802637 4148 tablet_replica.cc:330] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278: stopping tablet replica
I20260812 03:14:11.802690 4147 tablet_replica.cc:330] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c: stopping tablet replica
I20260812 03:14:11.803030 4148 raft_consensus.cc:2229] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:14:11.803023 4147 raft_consensus.cc:2229] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:14:11.803309 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 3344
I20260812 03:14:11.803532 4147 raft_consensus.cc:2258] T eacf6dead0794be2ba611936cb51a5e6 P e840f035c4304a71b553987947cb889c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:14:11.804347 4148 raft_consensus.cc:2258] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:14:11.812556 4148 ts_tablet_manager.cc:1905] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:14:11.816066 4148 ts_tablet_manager.cc:1918] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:14:11.816179 4148 log.cc:1196] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/eacf6dead0794be2ba611936cb51a5e6
I20260812 03:14:11.816510 4148 ts_tablet_manager.cc:1939] T eacf6dead0794be2ba611936cb51a5e6 P 24352d182e984c069bcb5eee8e3d5278: Deleting consensus metadata
I20260812 03:14:11.816484 4150 tablet_replica.cc:330] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2: stopping tablet replica
I20260812 03:14:11.816928 4150 raft_consensus.cc:2229] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:14:11.817422 4150 raft_consensus.cc:2258] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:14:11.818710 3209 catalog_manager.cc:4841] TS 24352d182e984c069bcb5eee8e3d5278 (127.16.217.67:37289): tablet eacf6dead0794be2ba611936cb51a5e6 (table test-table [id=0a62bdd87e894c5192e49e5e5feaca2f]) successfully deleted
I20260812 03:14:11.819283 4150 ts_tablet_manager.cc:1905] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:14:11.821997 4150 ts_tablet_manager.cc:1918] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:14:11.822120 4150 log.cc:1196] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/eacf6dead0794be2ba611936cb51a5e6
I20260812 03:14:11.822501 4150 ts_tablet_manager.cc:1939] T eacf6dead0794be2ba611936cb51a5e6 P 898be7030e914b399fe5f63b792376d2: Deleting consensus metadata
W20260812 03:14:11.823119 3208 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.217.65:36187 (error 108)
I20260812 03:14:11.824221 3209 catalog_manager.cc:4841] TS 898be7030e914b399fe5f63b792376d2 (127.16.217.66:36025): tablet eacf6dead0794be2ba611936cb51a5e6 (table test-table [id=0a62bdd87e894c5192e49e5e5feaca2f]) successfully deleted
W20260812 03:14:11.824376 4098 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:14:11.823982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:36187 (local address 127.0.0.1:48902)
0812 03:14:11.824163 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:11.824258 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:36187: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":74}
W20260812 03:14:11.824570 3208 catalog_manager.cc:4584] TS e840f035c4304a71b553987947cb889c (127.16.217.65:36187): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eacf6dead0794be2ba611936cb51a5e6: Network error: Client connection negotiation failed: client connection to 127.16.217.65:36187: connect: Connection reset by peer (error 104)
I20260812 03:14:11.824637 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 3588
I20260812 03:14:11.873896 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 3850
I20260812 03:14:11.880945 4098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.879695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:36187 (local address 127.0.0.1:48914)
0812 03:14:11.880802 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:11.880863 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1018}
I20260812 03:14:11.882210 4098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:14:11.881902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:36187 (local address 127.0.0.1:48926)
0812 03:14:11.882066 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 03:14:11.882122 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260812 03:14:11.892402 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 3179
2026-08-12T03:14:11Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2703 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:14:11 dist-test-slave-7nm7 krb5kdc[4157](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:14:11 dist-test-slave-7nm7 krb5kdc[4157](info): set up 2 sockets
Aug 12 03:14:11 dist-test-slave-7nm7 krb5kdc[4157](info): commencing operation
krb5kdc: starting...
W20260812 03:14:13.211396 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:14:13 dist-test-slave-7nm7 krb5kdc[4157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:14:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:14:13Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.16.217.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.16.217.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.16.217.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/rangeradmin_127.16.217.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.16.217.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/rangeradmin_127.16.217.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.16.217.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.16.217.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.16.217.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/rangerlookup_127.16.217.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.16.217.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/rangerlookup_127.16.217.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.217.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.16.217.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.217.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/HTTP_127.16.217.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.217.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/HTTP_127.16.217.84@KRBTEST.COM.keytab.
I20260812 03:14:13.395231 17253 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskm8e4iX/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/postgres -l logfile start
2026-08-12 03:14:16.207 UTC [4210] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 03:14:16.207 UTC [4210] LOG: listening on IPv4 address "127.16.217.84", port 47259
2026-08-12 03:14:16.210 UTC [4210] LOG: listening on Unix socket "/tmp/.s.PGSQL.47259"
2026-08-12 03:14:16.233 UTC [4215] LOG: database system was shut down at 2026-08-12 03:14:14 UTC
2026-08-12 03:14:16.239 UTC [4210] LOG: database system is ready to accept connections
I20260812 03:14:17.481520 17253 mini_postgres.cc:96] Postgres bound to 47259
2026-08-12 03:14:17.490 UTC [4233] FATAL: database "slave" does not exist
127.16.217.84:47259 - accepting connections
I20260812 03:14:17.491493 17253 mini_ranger.cc:161] Starting Ranger...
I20260812 03:14:17.508616 17253 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 03:14:18.026588 17253 mini_ranger.cc:87] Created ranger Postgres database
I20260812 03:14:18.026674 17253 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ranger-admin
2026-08-12 03:14:18,504 [I] DB FLAVOR :POSTGRES
2026-08-12 03:14:18,505 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:14:18,505 [I] Checking connection..
2026-08-12 03:14:18,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 03:14:18,804 [I] Checking connection passed.
2026-08-12 03:14:18,805 [I] --------- Verifying version history table ---------
2026-08-12 03:14:18,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:14:19,072 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:14:19,073 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:14:19,333 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:14:19,333 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 03:14:19,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 03:14:19.594 UTC [4346] WARNING: there is no transaction in progress
2026-08-12 03:14:19,604 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 03:14:19,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:14:19,890 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 03:14:19,891 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:14:19,891 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 03:14:20,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 03:14:20,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7nm7.c.gcp-upstream.internal','N') ;"
2026-08-12 03:14:20,702 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:14:20,982 [I] Table x_portal_user does not exist in database ranger
2026-08-12 03:14:20,982 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:14:21,269 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 03:14:21,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:14:21,534 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:14:21,808 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 03:14:21,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 03:14:22.902 UTC [4527] WARNING: there is no transaction in progress
2026-08-12 03:14:22.920 UTC [4527] WARNING: there is no transaction in progress
2026-08-12 03:14:22.938 UTC [4527] WARNING: there is no transaction in progress
2026-08-12 03:14:22.955 UTC [4527] WARNING: there is no transaction in progress
2026-08-12 03:14:22.975 UTC [4527] WARNING: there is no transaction in progress
2026-08-12 03:14:22.998 UTC [4527] WARNING: there is no transaction in progress
2026-08-12 03:14:23.072 UTC [4527] WARNING: there is no transaction in progress
2026-08-12 03:14:23.081 UTC [4527] WARNING: there is no transaction in progress
2026-08-12 03:14:23.100 UTC [4527] WARNING: there is no transaction in progress
2026-08-12 03:14:23.113 UTC [4527] WARNING: there is no transaction in progress
2026-08-12 03:14:23.126 UTC [4527] WARNING: there is no transaction in progress
2026-08-12 03:14:23.141 UTC [4527] WARNING: there is no transaction in progress
2026-08-12 03:14:23.153 UTC [4527] WARNING: there is no transaction in progress
2026-08-12 03:14:23.161 UTC [4527] WARNING: there is no transaction in progress
2026-08-12 03:14:23.169 UTC [4527] WARNING: there is no transaction in progress
2026-08-12 03:14:23.177 UTC [4527] WARNING: there is no transaction in progress
2026-08-12 03:14:23,938 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 03:14:23,939 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 03:14:24,224 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 03:14:24,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:14:24,512 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 03:14:24,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:14:24,810 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 03:14:24,810 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:14:25,076 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:14:25,352 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7nm7.c.gcp-upstream.internal';"
2026-08-12 03:14:25,603 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 03:14:25,603 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 03:14:25,852 [I] DB_PATCHES have already been applied
I20260812 03:14:25.857933 17253 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ranger-admin:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskm8e4iX/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 03:14:25.858000 17253 mini_ranger.cc:193] Using host: 127.16.217.84
I20260812 03:14:25.862440 17253 mini_ranger.cc:227] Ranger admin URL: http://127.16.217.84:55775
Aug 12, 2026 3:14:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:14:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:14:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 3:14:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 3:14:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 3:14:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 3:14:26 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 03:14:27.345148 17253 mini_ranger.cc:160] Time spent starting Ranger: real 9.854s user 0.006s sys 0.006s
Aug 12, 2026 3:14:50 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 3:14:50 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 3:14:50 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 3:14:50 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 3:14:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 3:14:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 3:14:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 3:14:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 3:14:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 3:14:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 3:14:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 3:14:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 3:14:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 3:14:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 3:14:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 3:14:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 3:14:50 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 3:14:52 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 03:14:52.755287 17253 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.16.217.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.16.217.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.16.217.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/rangerkms_127.16.217.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.16.217.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/rangerkms_127.16.217.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.217.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.16.217.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.217.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/HTTP_127.16.217.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.217.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/HTTP_127.16.217.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 03:14:52 dist-test-slave-7nm7 krb5kdc[4157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504492, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 03:14:52.857828 17253 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 03:14:52.871400 17253 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 03:14:53.853087 17253 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 03:14:53.853199 17253 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ranger-kms
I20260812 03:14:53.853322 17253 mini_ranger_kms.cc:227] Using postgres at 127.16.217.84:47259
2026-08-12 03:14:54,316 [I] DB FLAVOR :POSTGRES
2026-08-12 03:14:54,316 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:14:54,316 [I] Checking connection
2026-08-12 03:14:54,316 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 03:14:54,633 [I] connection success
2026-08-12 03:14:54,634 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 03:14:54,634 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 03:14:54,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 03:14:54,902 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 03:14:54,902 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:14:54,902 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 03:14:54,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:47259/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 03:14:55,191 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 03:14:55.763579 17253 mini_ranger_kms.cc:310] Created kms service
I20260812 03:14:55.990693 17253 mini_ranger_kms.cc:326] Created kudu user
I20260812 03:14:56.042198 17253 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 03:14:56 dist-test-slave-7nm7 krb5kdc[4157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504492, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
I20260812 03:14:56.483358 17253 mini_ranger_kms.cc:384] Added ranger policy
I20260812 03:14:56.483525 17253 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ranger-kms:/tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 03:14:56.483551 17253 mini_ranger_kms.cc:240] Using host: 127.16.217.84
I20260812 03:14:56.489358 17253 mini_ranger_kms.cc:276] Ranger KMS PID: 4850
I20260812 03:14:56.489603 17253 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.16.217.84:50131
Aug 12, 2026 3:14:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:14:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:14:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 3:14:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 3:14:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 3:14:58 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 03:14:58 dist-test-slave-7nm7 krb5kdc[4157](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786504498, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.217.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:14:59.493621 17253 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.636s user 0.007s sys 0.387s
I20260812 03:14:59.493772 17253 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 03:14:59.493883 17253 mini_ranger_kms.cc:401] 127.16.217.84:50131/kms/v1/keys
Aug 12 03:14:59 dist-test-slave-7nm7 krb5kdc[4157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504498, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.217.84@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
Aug 12 03:14:59 dist-test-slave-7nm7 krb5kdc[4157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504498, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.217.84@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
Aug 12 03:15:02 dist-test-slave-7nm7 krb5kdc[4157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:15:02.451356 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:45645
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:34383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:45645
--ranger_config_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.217.84:50131/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:15:02.595827 4949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:15:02.596139 4949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:15:02.596240 4949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:15:02.600519 4949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:15:02.600610 4949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:15:02.600644 4949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:15:02.600710 4949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:15:02.600750 4949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:15:02.606492 4949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:34383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/master-0/wal
--ranger_kms_url=127.16.217.84:50131/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:45645
--ranger_config_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:45645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:15:02.607635 4949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:15:02.608701 4949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:15:02.617969 4956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:02.617969 4955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:02.618466 4961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:15:02.618549 4949 server_base.cc:1034] running on GCE node
I20260812 03:15:02.619168 4949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:15:02.620373 4949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:15:02.621574 4949 hybrid_clock.cc:648] HybridClock initialized: now 1786504502621550 us; error 41 us; skew 500 ppm
Aug 12 03:15:02 dist-test-slave-7nm7 krb5kdc[4157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504502, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:15:02.629591 4949 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:15:02.630949 4949 webserver.cc:460] Webserver started at http://127.16.217.126:39987/ using document root <none> and password file <none>
I20260812 03:15:02.631718 4949 fs_manager.cc:359] Metadata directory not provided
I20260812 03:15:02.631806 4949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:15:02.632201 4949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:15:02 dist-test-slave-7nm7 krb5kdc[4157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504502, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
I20260812 03:15:02.693992 4949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "4ce1181053e54f25b3e089811740dd46"
format_stamp: "Formatted at 2026-08-12 03:15:02 on dist-test-slave-7nm7"
server_key: "b7e60c1d76c2bb5ef01f7feab041fbb2"
server_key_iv: "c80c75bfca6b09a5c81e11b479e06854"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:02.694720 4949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "4ce1181053e54f25b3e089811740dd46"
format_stamp: "Formatted at 2026-08-12 03:15:02 on dist-test-slave-7nm7"
server_key: "b7e60c1d76c2bb5ef01f7feab041fbb2"
server_key_iv: "c80c75bfca6b09a5c81e11b479e06854"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:02.698887 4949 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:15:02.719635 4969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:02.721148 4949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:15:02.721280 4949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "4ce1181053e54f25b3e089811740dd46"
format_stamp: "Formatted at 2026-08-12 03:15:02 on dist-test-slave-7nm7"
server_key: "b7e60c1d76c2bb5ef01f7feab041fbb2"
server_key_iv: "c80c75bfca6b09a5c81e11b479e06854"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:02.721385 4949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:15:02.751860 4949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:15:02.755244 4949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:15:02.755515 4949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:15:02.768863 4949 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:45645
I20260812 03:15:02.768911 5065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:45645 every 8 connection(s)
I20260812 03:15:02.770038 4949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:15:02.772894 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 4949
I20260812 03:15:02.773036 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:15:02.773362 17253 external_mini_cluster.cc:1428] Setting key 9dcc26375ce89174da3555c09a6bd198
I20260812 03:15:02.773365 5067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:15:02.782320 5067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46: Bootstrap starting.
I20260812 03:15:02.785763 5067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46: Neither blocks nor log segments found. Creating new log.
Aug 12 03:15:02 dist-test-slave-7nm7 krb5kdc[4157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:15:02.786724 5067 log.cc:824] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46: Log is configured to *not* fsync() on all Append() calls
I20260812 03:15:02.789181 5067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46: No bootstrap required, opened a new log
I20260812 03:15:02.793504 5067 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce1181053e54f25b3e089811740dd46" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 45645 } }
I20260812 03:15:02.793761 5067 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:15:02.793845 5067 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ce1181053e54f25b3e089811740dd46, State: Initialized, Role: FOLLOWER
I20260812 03:15:02.794414 5076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:02.775004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34692 (local address 127.16.217.126:45645)
0812 03:15:02.775817 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:02.775884 (+ 67us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:02.775934 (+ 50us) server_negotiation.cc:407] Connection header received
0812 03:15:02.777407 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:02.777436 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:02.777953 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:02.778441 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:15:02.779623 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:02.781365 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:02.782624 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:02.783426 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:02.783611 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:02.787276 (+ 3665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:15:02.787284 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:15:02.787305 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:15:02.787343 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:15:02.790533 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:02.791154 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:02.791161 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:02.791168 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:02.791313 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:02.791661 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:02.791665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:02.791667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:02.792225 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:15:02.792438 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:02.792873 (+ 435us) server_negotiation.cc:299] Negotiation successful
0812 03:15:02.793187 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":317,"threads_started":1}
I20260812 03:15:02.794414 5067 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce1181053e54f25b3e089811740dd46" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 45645 } }
I20260812 03:15:02.794749 5067 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:15:02.794821 5067 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:15:02.794919 5067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:15:02.795897 5067 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce1181053e54f25b3e089811740dd46" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 45645 } }
I20260812 03:15:02.796259 5067 leader_election.cc:302] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ce1181053e54f25b3e089811740dd46; no voters:
I20260812 03:15:02.796648 5067 leader_election.cc:288] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:15:02.796936 5078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:15:02.797626 5078 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46 [term 1 LEADER]: Becoming Leader. State: Replica: 4ce1181053e54f25b3e089811740dd46, State: Running, Role: LEADER
I20260812 03:15:02.798004 5067 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:15:02.798002 5078 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce1181053e54f25b3e089811740dd46" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 45645 } }
I20260812 03:15:02.800092 5080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ce1181053e54f25b3e089811740dd46. Latest consensus state: current_term: 1 leader_uuid: "4ce1181053e54f25b3e089811740dd46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce1181053e54f25b3e089811740dd46" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 45645 } } }
I20260812 03:15:02.800261 5080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46 [sys.catalog]: This master's current role is: LEADER
I20260812 03:15:02.800774 5096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:15:02.801410 5067 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 03:15:02.800772 5081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ce1181053e54f25b3e089811740dd46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce1181053e54f25b3e089811740dd46" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 45645 } } }
I20260812 03:15:02.801807 5081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46 [sys.catalog]: This master's current role is: LEADER
I20260812 03:15:02.810472 5096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:15:02.820209 5096 catalog_manager.cc:1347] Generated new cluster ID: c214c518d690480c8d51aa422bab6f0a
I20260812 03:15:02.820299 5096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:15:02.842381 5096 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:15:02.843475 5096 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:15:02.851287 5096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46: Generated new TSK 0
I20260812 03:15:02.852069 5096 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
I20260812 03:15:03.778453 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.217.84:50131/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:45645
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:34383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:15:03.931005 5154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:15:03.931274 5154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:15:03.931317 5154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:15:03.935617 5154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:15:03.935678 5154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:15:03.935766 5154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:15:03.941540 5154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:34383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.16.217.84:50131/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:45645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:15:03.942773 5154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:15:03.943945 5154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:15:03.955590 5165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:03.955590 5164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:03.955986 5167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:15:03.957298 5154 server_base.cc:1034] running on GCE node
I20260812 03:15:03.957803 5154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:15:03.958729 5154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:15:03.959920 5154 hybrid_clock.cc:648] HybridClock initialized: now 1786504503959886 us; error 54 us; skew 500 ppm
Aug 12 03:15:03 dist-test-slave-7nm7 krb5kdc[4157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504503, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:15:03.963837 5154 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:15:03.965059 5154 webserver.cc:460] Webserver started at http://127.16.217.65:35805/ using document root <none> and password file <none>
I20260812 03:15:03.965963 5154 fs_manager.cc:359] Metadata directory not provided
I20260812 03:15:03.966084 5154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:15:03.966473 5154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:15:03 dist-test-slave-7nm7 krb5kdc[4157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504503, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
I20260812 03:15:03.986471 5154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "38c4b4b2d2d8495680bc893d6e9b343e"
format_stamp: "Formatted at 2026-08-12 03:15:03 on dist-test-slave-7nm7"
server_key: "58b13d74b76300329613b727b4d065b7"
server_key_iv: "8b1e09484107849b221a9e41935cbd63"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:03.987267 5154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "38c4b4b2d2d8495680bc893d6e9b343e"
format_stamp: "Formatted at 2026-08-12 03:15:03 on dist-test-slave-7nm7"
server_key: "58b13d74b76300329613b727b4d065b7"
server_key_iv: "8b1e09484107849b221a9e41935cbd63"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:03.995859 5154 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.001s sys 0.003s
I20260812 03:15:04.018782 5182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:04.020715 5154 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.001s
I20260812 03:15:04.020875 5154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "38c4b4b2d2d8495680bc893d6e9b343e"
format_stamp: "Formatted at 2026-08-12 03:15:03 on dist-test-slave-7nm7"
server_key: "58b13d74b76300329613b727b4d065b7"
server_key_iv: "8b1e09484107849b221a9e41935cbd63"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:04.021013 5154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:15:04.041625 5154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:15:04.045722 5154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:15:04.045987 5154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:15:04.047050 5154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:15:04.048317 5154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:15:04.048434 5154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:04.048600 5154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:15:04.048693 5154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:04.065471 5154 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:43025
I20260812 03:15:04.067119 5154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:15:04.068590 5380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:43025 every 8 connection(s)
I20260812 03:15:04.076972 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 5154
I20260812 03:15:04.077111 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:15:04.077448 17253 external_mini_cluster.cc:1428] Setting key 729b175e9d492a18bc399d0d9efa4f9d
Aug 12 03:15:04 dist-test-slave-7nm7 krb5kdc[4157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504503, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.217.66@KRBTEST.COM; defaulting to no policy
I20260812 03:15:04.095984 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:04.073304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:48827 (local address 127.16.217.126:45645)
0812 03:15:04.074415 (+ 1111us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:04.074421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:04.074439 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:15:04.075857 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:04.075860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:04.075927 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:04.076020 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:15:04.081912 (+ 5892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:04.083142 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:04.085313 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:04.085736 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:04.085837 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:04.090256 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:15:04.090265 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:15:04.090270 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:15:04.090308 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:15:04.092633 (+ 2325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:04.093410 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:04.093416 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:04.093418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:04.093520 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:04.094039 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:04.094043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:04.094044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:04.094375 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:15:04.094476 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:04.095457 (+ 981us) server_negotiation.cc:299] Negotiation successful
0812 03:15:04.095665 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1027,"thread_start_us":139,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.66@KRBTEST.COM" created.
I20260812 03:15:04.098153 5385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:04.071876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:45645 (local address 127.16.217.65:48827)
0812 03:15:04.073973 (+ 2097us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:04.074036 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:04.075465 (+ 1429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:04.076353 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:04.076363 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:04.077029 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:15:04.081572 (+ 4543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:04.081599 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:04.083507 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:04.083513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:04.085144 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:04.085161 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:04.086110 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:04.086116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:04.086483 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:04.087481 (+ 998us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:15:04.087510 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:15:04.089873 (+ 2363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:15:04.092811 (+ 2938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:15:04.092820 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:15:04.092841 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:15:04.093268 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:15:04.093654 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:15:04.093660 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:15:04.093663 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:15:04.093909 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:15:04.094562 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:15:04.094572 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:15:04.095030 (+ 458us) client_negotiation.cc:769] Sending connection context
0812 03:15:04.096512 (+ 1482us) client_negotiation.cc:240] Negotiation successful
0812 03:15:04.096939 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1850,"thread_start_us":306,"threads_started":1}
I20260812 03:15:04.100126 5381 heartbeater.cc:347] Connected to a master server at 127.16.217.126:45645
I20260812 03:15:04.100412 5381 heartbeater.cc:464] Registering TS with master...
I20260812 03:15:04.101091 5381 heartbeater.cc:511] Master 127.16.217.126:45645 requested a full tablet report, sending...
I20260812 03:15:04.103370 5003 ts_manager.cc:194] Registered new tserver with Master: 38c4b4b2d2d8495680bc893d6e9b343e (127.16.217.65:43025)
I20260812 03:15:04.105168 5003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:48827
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
I20260812 03:15:04.122000 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.217.84:50131/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:45645
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:34383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:15:04 dist-test-slave-7nm7 krb5kdc[4157](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786504504, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 03:15:04.333674 5392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:15:04.333981 5392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:15:04.334054 5392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:15:04.338521 5392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:15:04.338594 5392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:15:04.338658 5392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:15:04.345237 5392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:34383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.16.217.84:50131/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:45645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:15:04.346580 5392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:15:04.347707 5392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:15:04.362295 5401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:15:04.362725 5392 server_base.cc:1034] running on GCE node
W20260812 03:15:04.362733 5403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:04.363019 5400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:15:04.363498 5392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:15:04.364223 5392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:15:04.365429 5392 hybrid_clock.cc:648] HybridClock initialized: now 1786504504365400 us; error 48 us; skew 500 ppm
Aug 12 03:15:04 dist-test-slave-7nm7 krb5kdc[4157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504504, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:15:04.369750 5392 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:15:04.371191 5392 webserver.cc:460] Webserver started at http://127.16.217.66:34593/ using document root <none> and password file <none>
I20260812 03:15:04.371840 5392 fs_manager.cc:359] Metadata directory not provided
I20260812 03:15:04.371946 5392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:15:04.372197 5392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:15:04 dist-test-slave-7nm7 krb5kdc[4157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504504, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
I20260812 03:15:04.388195 5392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "f3c28788214e478fb374421aeb11acdd"
format_stamp: "Formatted at 2026-08-12 03:15:04 on dist-test-slave-7nm7"
server_key: "a1676944dabc6e1b6cac705f862b8b47"
server_key_iv: "89d84bbf0d5d08eb6f591300acef5665"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:04.389520 5392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "f3c28788214e478fb374421aeb11acdd"
format_stamp: "Formatted at 2026-08-12 03:15:04 on dist-test-slave-7nm7"
server_key: "a1676944dabc6e1b6cac705f862b8b47"
server_key_iv: "89d84bbf0d5d08eb6f591300acef5665"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:04.399430 5392 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.005s sys 0.001s
I20260812 03:15:04.422925 5412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:04.425052 5392 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.002s sys 0.001s
I20260812 03:15:04.425290 5392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "f3c28788214e478fb374421aeb11acdd"
format_stamp: "Formatted at 2026-08-12 03:15:04 on dist-test-slave-7nm7"
server_key: "a1676944dabc6e1b6cac705f862b8b47"
server_key_iv: "89d84bbf0d5d08eb6f591300acef5665"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:04.425613 5392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:15:04.440793 5392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:15:04.456089 5392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:15:04.456350 5392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:15:04.457199 5392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:15:04.458576 5392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:15:04.458664 5392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:04.458747 5392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:15:04.458786 5392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:04.488454 5392 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:35255
I20260812 03:15:04.492069 5610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:35255 every 8 connection(s)
I20260812 03:15:04.493011 5392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:15:04.501706 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 5392
I20260812 03:15:04.501863 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:15:04.502192 17253 external_mini_cluster.cc:1428] Setting key 8b4d436ef096443146865a75ac01a16d
Aug 12 03:15:04 dist-test-slave-7nm7 krb5kdc[4157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504504, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:15:04.512959 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:04.495638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:51275 (local address 127.16.217.126:45645)
0812 03:15:04.495785 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:04.495790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:04.497152 (+ 1362us) server_negotiation.cc:407] Connection header received
0812 03:15:04.498918 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:04.498922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:04.498984 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:04.499044 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:15:04.500986 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:04.501820 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:04.504074 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:04.504423 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:04.504504 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:04.507892 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:15:04.507896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:15:04.507899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:15:04.507929 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:15:04.510008 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:04.510886 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:04.510892 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:04.510893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:04.511021 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:04.511493 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:04.511498 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:04.511499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:04.511807 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:15:04.511905 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:04.512618 (+ 713us) server_negotiation.cc:299] Negotiation successful
0812 03:15:04.512775 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:15:04.513990 5614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:04.496175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:45645 (local address 127.16.217.66:51275)
0812 03:15:04.496977 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:04.497039 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:04.498640 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:04.499483 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:04.499492 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:04.499948 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:15:04.500806 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:04.500829 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:04.501997 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:04.502000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:04.503935 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:04.503947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:04.504744 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:04.504748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:04.504953 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:04.505708 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:15:04.505731 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:15:04.507680 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:15:04.510144 (+ 2464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:15:04.510151 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:15:04.510170 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:15:04.510609 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:15:04.511139 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:15:04.511143 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:15:04.511145 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:15:04.511346 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:15:04.511930 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:15:04.511937 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:15:04.512292 (+ 355us) client_negotiation.cc:769] Sending connection context
0812 03:15:04.512507 (+ 215us) client_negotiation.cc:240] Negotiation successful
0812 03:15:04.512856 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":195,"threads_started":1}
I20260812 03:15:04.515570 5612 heartbeater.cc:347] Connected to a master server at 127.16.217.126:45645
I20260812 03:15:04.515848 5612 heartbeater.cc:464] Registering TS with master...
I20260812 03:15:04.516592 5612 heartbeater.cc:511] Master 127.16.217.126:45645 requested a full tablet report, sending...
I20260812 03:15:04.517978 5003 ts_manager.cc:194] Registered new tserver with Master: f3c28788214e478fb374421aeb11acdd (127.16.217.66:35255)
I20260812 03:15:04.518951 5003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:51275
WARNING: no policy specified for kudu/127.16.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
I20260812 03:15:04.564241 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.217.84:50131/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:45645
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:34383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:15:04.764925 5621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:15:04.765437 5621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:15:04.765591 5621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:15:04.771837 5621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:15:04.771966 5621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:15:04.772104 5621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:15:04.780417 5621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:34383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.16.217.84:50131/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:45645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:15:04.782261 5621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:15:04.783757 5621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:15:04.800606 5634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:04.801043 5630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:04.801637 5628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:15:04.806648 5621 server_base.cc:1034] running on GCE node
I20260812 03:15:04.807407 5621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:15:04.808331 5621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:15:04.811780 5621 hybrid_clock.cc:648] HybridClock initialized: now 1786504504811658 us; error 136 us; skew 500 ppm
Aug 12 03:15:04 dist-test-slave-7nm7 krb5kdc[4157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504504, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:15:04.815675 5621 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:15:04.817020 5621 webserver.cc:460] Webserver started at http://127.16.217.67:44013/ using document root <none> and password file <none>
I20260812 03:15:04.817831 5621 fs_manager.cc:359] Metadata directory not provided
I20260812 03:15:04.817979 5621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:15:04.818522 5621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:15:04 dist-test-slave-7nm7 krb5kdc[4157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504504, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
I20260812 03:15:04.837195 5621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "ae1174bf486d496e83d70f723e38cdaf"
format_stamp: "Formatted at 2026-08-12 03:15:04 on dist-test-slave-7nm7"
server_key: "6bd7650489604220988a806835342bd6"
server_key_iv: "e376c22db5823b5d2b6b79ebfc57422f"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:04.839133 5621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "ae1174bf486d496e83d70f723e38cdaf"
format_stamp: "Formatted at 2026-08-12 03:15:04 on dist-test-slave-7nm7"
server_key: "6bd7650489604220988a806835342bd6"
server_key_iv: "e376c22db5823b5d2b6b79ebfc57422f"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:04.844630 5621 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
Aug 12 03:15:04 dist-test-slave-7nm7 krb5kdc[4157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504504, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
I20260812 03:15:04.869225 5644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:04.870950 5621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:15:04.871095 5621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "ae1174bf486d496e83d70f723e38cdaf"
format_stamp: "Formatted at 2026-08-12 03:15:04 on dist-test-slave-7nm7"
server_key: "6bd7650489604220988a806835342bd6"
server_key_iv: "e376c22db5823b5d2b6b79ebfc57422f"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:04.871243 5621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:15:04.906831 5621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:15:04.911545 5621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:15:04.911818 5621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:15:04.912568 5621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:15:04.914311 5621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:15:04.914430 5621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:04.914542 5621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:15:04.914578 5621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:04.930575 5621 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:34589
I20260812 03:15:04.931767 5621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:15:04.932706 5843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:34589 every 8 connection(s)
I20260812 03:15:04.941777 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 5621
I20260812 03:15:04.941929 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:15:04.942324 17253 external_mini_cluster.cc:1428] Setting key 41fd4f2ea34a680ab2a0aa421f1e01fc
Aug 12 03:15:04 dist-test-slave-7nm7 krb5kdc[4157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504504, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:15:04.949699 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:04.933884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:58133 (local address 127.16.217.126:45645)
0812 03:15:04.934055 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:04.934060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:04.935117 (+ 1057us) server_negotiation.cc:407] Connection header received
0812 03:15:04.936491 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:04.936496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:04.936558 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:04.936620 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:15:04.938193 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:04.939139 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:04.941004 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:04.941363 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:04.941452 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:04.944926 (+ 3474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:15:04.944931 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:15:04.944934 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:15:04.944963 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:15:04.946819 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:04.947498 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:04.947501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:04.947503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:04.947592 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:04.948163 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:04.948168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:04.948170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:04.948468 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:15:04.948545 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:04.949387 (+ 842us) server_negotiation.cc:299] Negotiation successful
0812 03:15:04.949515 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 03:15:04.951750 5846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:04.934397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:45645 (local address 127.16.217.67:58133)
0812 03:15:04.934954 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:04.935008 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:04.936165 (+ 1157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:04.936987 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:04.936994 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:04.937415 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:15:04.938023 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:04.938039 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:04.939308 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:04.939314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:04.940850 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:04.940865 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:04.941708 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:04.941714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:04.942075 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:04.942871 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:15:04.942893 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:15:04.944701 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:15:04.946972 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:15:04.946979 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:15:04.946998 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:15:04.947375 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:15:04.947742 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:15:04.947745 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:15:04.947747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:15:04.947989 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:15:04.948631 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:15:04.948638 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:15:04.949076 (+ 438us) client_negotiation.cc:769] Sending connection context
0812 03:15:04.950199 (+ 1123us) client_negotiation.cc:240] Negotiation successful
0812 03:15:04.950591 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":210,"threads_started":1}
I20260812 03:15:04.953466 5844 heartbeater.cc:347] Connected to a master server at 127.16.217.126:45645
I20260812 03:15:04.953753 5844 heartbeater.cc:464] Registering TS with master...
I20260812 03:15:04.954417 5844 heartbeater.cc:511] Master 127.16.217.126:45645 requested a full tablet report, sending...
I20260812 03:15:04.955698 4999 ts_manager.cc:194] Registered new tserver with Master: ae1174bf486d496e83d70f723e38cdaf (127.16.217.67:34589)
I20260812 03:15:04.956396 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:15:04.956492 4999 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:58133
I20260812 03:15:04.973172 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:04.962821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52246 (local address 127.16.217.126:45645)
0812 03:15:04.962967 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:04.962972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:04.963415 (+ 443us) server_negotiation.cc:407] Connection header received
0812 03:15:04.963638 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:04.963641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:04.963695 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:04.963747 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:15:04.964806 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:04.965644 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:04.966935 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:04.967300 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:04.967376 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:04.969266 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:15:04.969270 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:15:04.969273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:15:04.969302 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:15:04.971128 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:04.971696 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:04.971700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:04.971702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:04.971809 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:04.972116 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:04.972119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:04.972121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:04.972341 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:15:04.972445 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:04.972856 (+ 411us) server_negotiation.cc:299] Negotiation successful
0812 03:15:04.972991 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:15:04.977612 4999 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52246:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:15:04.982183 4999 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:15:05.015386 5869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:04.997511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:43025 (local address 127.0.0.1:36922)
0812 03:15:05.002480 (+ 4969us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:05.002496 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:05.002602 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:05.007105 (+ 4503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:05.007108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:05.007130 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:15:05.007234 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:05.007240 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.008706 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.008710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:05.009929 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:05.009941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.014618 (+ 4677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.014623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:05.014786 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:05.014845 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 03:15:05.014993 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 03:15:05.015199 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4903,"thread_start_us":109,"threads_started":1}
I20260812 03:15:05.016098 5876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:04.998477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36922 (local address 127.16.217.65:43025)
0812 03:15:04.999444 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:04.999450 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:05.002711 (+ 3261us) server_negotiation.cc:407] Connection header received
0812 03:15:05.002801 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:05.002828 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:05.003253 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:05.003418 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:15:05.007378 (+ 3960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.008538 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.010087 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.011040 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.011162 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:05.014534 (+ 3372us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:05.015800 (+ 1266us) server_negotiation.cc:299] Negotiation successful
0812 03:15:05.015935 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":869,"thread_start_us":123,"threads_started":1}
I20260812 03:15:05.023628 5227 tablet_service.cc:1467] Processing CreateTablet for tablet a46d3c0595e84950bf4fea30a1e1d30f (DEFAULT_TABLE table=test-table [id=64e4b18d0e5d4e8b8349108a30deac97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:15:05.025198 5227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a46d3c0595e84950bf4fea30a1e1d30f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:15:05.017498 5871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:04.997181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:35255 (local address 127.0.0.1:56392)
0812 03:15:04.999904 (+ 2723us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:04.999916 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:04.999995 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:05.001230 (+ 1235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:05.001237 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:05.001263 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:15:05.001401 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:05.001408 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.005486 (+ 4078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.005490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:05.006961 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:05.006974 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.017054 (+ 10080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.017058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:05.017175 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:05.017201 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 03:15:05.017277 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 03:15:05.017354 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2664,"thread_start_us":101,"threads_started":1}
I20260812 03:15:05.032076 5874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:04.998107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56392 (local address 127.16.217.66:35255)
0812 03:15:04.999248 (+ 1141us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:04.999266 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:05.000217 (+ 951us) server_negotiation.cc:407] Connection header received
0812 03:15:05.000468 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:05.000476 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:05.000836 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:05.001023 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:15:05.001550 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.005330 (+ 3780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.013273 (+ 7943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.016686 (+ 3413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.016817 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:05.016975 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:05.031755 (+ 14780us) server_negotiation.cc:299] Negotiation successful
0812 03:15:05.031914 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1035,"thread_start_us":115,"threads_started":1}
I20260812 03:15:05.040369 5500 tablet_service.cc:1467] Processing CreateTablet for tablet a46d3c0595e84950bf4fea30a1e1d30f (DEFAULT_TABLE table=test-table [id=64e4b18d0e5d4e8b8349108a30deac97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:15:05.042367 5500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a46d3c0595e84950bf4fea30a1e1d30f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:15:05.050995 5873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:04.995781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:34589 (local address 127.0.0.1:60638)
0812 03:15:04.999014 (+ 3233us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:04.999513 (+ 499us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:04.999661 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:05.004391 (+ 4730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:05.004395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:05.004415 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:15:05.004522 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:05.004528 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.017817 (+ 13289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.017821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:05.019060 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:05.019074 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.050484 (+ 31410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.050490 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:05.050631 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:05.050669 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 03:15:05.050733 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:15:05.050826 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3033,"thread_start_us":118,"threads_started":1}
I20260812 03:15:05.052105 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:04.996685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60638 (local address 127.16.217.67:34589)
0812 03:15:05.003766 (+ 7081us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:05.003779 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:05.003803 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:15:05.003879 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:05.003884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:05.004074 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:05.004232 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:15:05.011526 (+ 7294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.012921 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.019298 (+ 6377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.034619 (+ 15321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.051554 (+ 16935us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:05.051765 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:05.051830 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 03:15:05.051942 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6957,"thread_start_us":155,"threads_started":1}
I20260812 03:15:05.055759 5692 tablet_service.cc:1467] Processing CreateTablet for tablet a46d3c0595e84950bf4fea30a1e1d30f (DEFAULT_TABLE table=test-table [id=64e4b18d0e5d4e8b8349108a30deac97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:15:05.057066 5692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a46d3c0595e84950bf4fea30a1e1d30f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:15:05.069936 5881 tablet_bootstrap.cc:492] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf: Bootstrap starting.
I20260812 03:15:05.076804 5882 tablet_bootstrap.cc:492] T a46d3c0595e84950bf4fea30a1e1d30f P 38c4b4b2d2d8495680bc893d6e9b343e: Bootstrap starting.
I20260812 03:15:05.080227 5882 tablet_bootstrap.cc:654] T a46d3c0595e84950bf4fea30a1e1d30f P 38c4b4b2d2d8495680bc893d6e9b343e: Neither blocks nor log segments found. Creating new log.
I20260812 03:15:05.080797 5883 tablet_bootstrap.cc:492] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd: Bootstrap starting.
I20260812 03:15:05.081462 5881 tablet_bootstrap.cc:654] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf: Neither blocks nor log segments found. Creating new log.
I20260812 03:15:05.083976 5883 tablet_bootstrap.cc:654] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd: Neither blocks nor log segments found. Creating new log.
I20260812 03:15:05.085043 5881 log.cc:824] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf: Log is configured to *not* fsync() on all Append() calls
I20260812 03:15:05.087082 5882 log.cc:824] T a46d3c0595e84950bf4fea30a1e1d30f P 38c4b4b2d2d8495680bc893d6e9b343e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:15:05.088035 5883 log.cc:824] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd: Log is configured to *not* fsync() on all Append() calls
I20260812 03:15:05.091356 5881 tablet_bootstrap.cc:492] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf: No bootstrap required, opened a new log
I20260812 03:15:05.091691 5881 ts_tablet_manager.cc:1397] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf: Time spent bootstrapping tablet: real 0.022s user 0.008s sys 0.000s
I20260812 03:15:05.097759 5881 raft_consensus.cc:348] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1174bf486d496e83d70f723e38cdaf" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34589 } } peers { permanent_uuid: "f3c28788214e478fb374421aeb11acdd" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35255 } } peers { permanent_uuid: "38c4b4b2d2d8495680bc893d6e9b343e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 43025 } }
I20260812 03:15:05.100623 5882 tablet_bootstrap.cc:492] T a46d3c0595e84950bf4fea30a1e1d30f P 38c4b4b2d2d8495680bc893d6e9b343e: No bootstrap required, opened a new log
I20260812 03:15:05.100921 5882 ts_tablet_manager.cc:1397] T a46d3c0595e84950bf4fea30a1e1d30f P 38c4b4b2d2d8495680bc893d6e9b343e: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.000s
I20260812 03:15:05.108621 5381 heartbeater.cc:503] Master 127.16.217.126:45645 was elected leader, sending a full tablet report...
I20260812 03:15:05.108796 5883 tablet_bootstrap.cc:492] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd: No bootstrap required, opened a new log
I20260812 03:15:05.109081 5883 ts_tablet_manager.cc:1397] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd: Time spent bootstrapping tablet: real 0.028s user 0.006s sys 0.000s
I20260812 03:15:05.114619 5881 raft_consensus.cc:374] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:15:05.114822 5881 raft_consensus.cc:729] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae1174bf486d496e83d70f723e38cdaf, State: Initialized, Role: FOLLOWER
I20260812 03:15:05.116250 5882 raft_consensus.cc:348] T a46d3c0595e84950bf4fea30a1e1d30f P 38c4b4b2d2d8495680bc893d6e9b343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1174bf486d496e83d70f723e38cdaf" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34589 } } peers { permanent_uuid: "f3c28788214e478fb374421aeb11acdd" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35255 } } peers { permanent_uuid: "38c4b4b2d2d8495680bc893d6e9b343e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 43025 } }
I20260812 03:15:05.117266 5881 consensus_queue.cc:260] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1174bf486d496e83d70f723e38cdaf" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34589 } } peers { permanent_uuid: "f3c28788214e478fb374421aeb11acdd" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35255 } } peers { permanent_uuid: "38c4b4b2d2d8495680bc893d6e9b343e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 43025 } }
I20260812 03:15:05.119365 5881 ts_tablet_manager.cc:1428] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf: Time spent starting tablet: real 0.028s user 0.002s sys 0.005s
I20260812 03:15:05.118182 5883 raft_consensus.cc:348] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1174bf486d496e83d70f723e38cdaf" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34589 } } peers { permanent_uuid: "f3c28788214e478fb374421aeb11acdd" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35255 } } peers { permanent_uuid: "38c4b4b2d2d8495680bc893d6e9b343e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 43025 } }
I20260812 03:15:05.119809 5883 raft_consensus.cc:374] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:15:05.119930 5882 raft_consensus.cc:374] T a46d3c0595e84950bf4fea30a1e1d30f P 38c4b4b2d2d8495680bc893d6e9b343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:15:05.120036 5883 raft_consensus.cc:729] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c28788214e478fb374421aeb11acdd, State: Initialized, Role: FOLLOWER
I20260812 03:15:05.120132 5882 raft_consensus.cc:729] T a46d3c0595e84950bf4fea30a1e1d30f P 38c4b4b2d2d8495680bc893d6e9b343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c4b4b2d2d8495680bc893d6e9b343e, State: Initialized, Role: FOLLOWER
I20260812 03:15:05.121040 5844 heartbeater.cc:503] Master 127.16.217.126:45645 was elected leader, sending a full tablet report...
I20260812 03:15:05.121407 5883 consensus_queue.cc:260] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1174bf486d496e83d70f723e38cdaf" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34589 } } peers { permanent_uuid: "f3c28788214e478fb374421aeb11acdd" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35255 } } peers { permanent_uuid: "38c4b4b2d2d8495680bc893d6e9b343e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 43025 } }
I20260812 03:15:05.121730 5882 consensus_queue.cc:260] T a46d3c0595e84950bf4fea30a1e1d30f P 38c4b4b2d2d8495680bc893d6e9b343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1174bf486d496e83d70f723e38cdaf" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34589 } } peers { permanent_uuid: "f3c28788214e478fb374421aeb11acdd" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35255 } } peers { permanent_uuid: "38c4b4b2d2d8495680bc893d6e9b343e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 43025 } }
I20260812 03:15:05.123397 5882 ts_tablet_manager.cc:1428] T a46d3c0595e84950bf4fea30a1e1d30f P 38c4b4b2d2d8495680bc893d6e9b343e: Time spent starting tablet: real 0.022s user 0.008s sys 0.000s
I20260812 03:15:05.124341 5883 ts_tablet_manager.cc:1428] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd: Time spent starting tablet: real 0.015s user 0.007s sys 0.000s
I20260812 03:15:05.126946 5612 heartbeater.cc:503] Master 127.16.217.126:45645 was elected leader, sending a full tablet report...
W20260812 03:15:05.189487 5845 tablet.cc:2369] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 03:15:05 dist-test-slave-7nm7 krb5kdc[4157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504504, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
W20260812 03:15:05.245824 5613 tablet.cc:2369] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:15:05.320657 5384 tablet.cc:2369] T a46d3c0595e84950bf4fea30a1e1d30f P 38c4b4b2d2d8495680bc893d6e9b343e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:15:05.342876 5888 raft_consensus.cc:482] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:15:05.343149 5888 raft_consensus.cc:504] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1174bf486d496e83d70f723e38cdaf" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34589 } } peers { permanent_uuid: "f3c28788214e478fb374421aeb11acdd" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35255 } } peers { permanent_uuid: "38c4b4b2d2d8495680bc893d6e9b343e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 43025 } }
I20260812 03:15:05.344589 5888 leader_election.cc:288] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae1174bf486d496e83d70f723e38cdaf (127.16.217.67:34589), 38c4b4b2d2d8495680bc893d6e9b343e (127.16.217.65:43025)
I20260812 03:15:05.353596 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:05.345410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:42073 (local address 127.16.217.67:34589)
0812 03:15:05.345542 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:05.345547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:05.348918 (+ 3371us) server_negotiation.cc:407] Connection header received
0812 03:15:05.349141 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:05.349144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:05.349209 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:05.349259 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:15:05.349691 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.350530 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.352234 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.352839 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.352941 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:05.352999 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:05.353365 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 03:15:05.353450 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:15:05.354146 5899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:05.344911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:34589 (local address 127.16.217.66:42073)
0812 03:15:05.348802 (+ 3891us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:05.348820 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:05.349012 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:05.349411 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:05.349414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:05.349433 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:15:05.349561 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:05.349567 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.350706 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.350709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:05.352062 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:05.352074 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.353112 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.353115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:05.353224 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:05.353248 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:15:05.353840 (+ 592us) client_negotiation.cc:240] Negotiation successful
0812 03:15:05.353924 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3784,"thread_start_us":156,"threads_started":1}
I20260812 03:15:05.355014 5757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a46d3c0595e84950bf4fea30a1e1d30f" candidate_uuid: "f3c28788214e478fb374421aeb11acdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae1174bf486d496e83d70f723e38cdaf" is_pre_election: true
I20260812 03:15:05.355405 5757 raft_consensus.cc:2454] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c28788214e478fb374421aeb11acdd in term 0.
I20260812 03:15:05.357026 5413 leader_election.cc:302] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae1174bf486d496e83d70f723e38cdaf, f3c28788214e478fb374421aeb11acdd; no voters:
I20260812 03:15:05.357426 5888 raft_consensus.cc:2781] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:15:05.357514 5888 raft_consensus.cc:482] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:15:05.357560 5888 raft_consensus.cc:3037] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:15:05.359400 5888 raft_consensus.cc:504] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1174bf486d496e83d70f723e38cdaf" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34589 } } peers { permanent_uuid: "f3c28788214e478fb374421aeb11acdd" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35255 } } peers { permanent_uuid: "38c4b4b2d2d8495680bc893d6e9b343e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 43025 } }
I20260812 03:15:05.359818 5888 leader_election.cc:288] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [CANDIDATE]: Term 1 election: Requested vote from peers ae1174bf486d496e83d70f723e38cdaf (127.16.217.67:34589), 38c4b4b2d2d8495680bc893d6e9b343e (127.16.217.65:43025)
I20260812 03:15:05.360452 5757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a46d3c0595e84950bf4fea30a1e1d30f" candidate_uuid: "f3c28788214e478fb374421aeb11acdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae1174bf486d496e83d70f723e38cdaf"
I20260812 03:15:05.360594 5757 raft_consensus.cc:3037] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:15:05.362154 5757 raft_consensus.cc:2454] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c28788214e478fb374421aeb11acdd in term 1.
I20260812 03:15:05.363675 5413 leader_election.cc:302] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae1174bf486d496e83d70f723e38cdaf, f3c28788214e478fb374421aeb11acdd; no voters:
I20260812 03:15:05.363894 5888 raft_consensus.cc:2781] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:15:05.364253 5888 raft_consensus.cc:686] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [term 1 LEADER]: Becoming Leader. State: Replica: f3c28788214e478fb374421aeb11acdd, State: Running, Role: LEADER
I20260812 03:15:05.364671 5888 consensus_queue.cc:237] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1174bf486d496e83d70f723e38cdaf" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34589 } } peers { permanent_uuid: "f3c28788214e478fb374421aeb11acdd" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35255 } } peers { permanent_uuid: "38c4b4b2d2d8495680bc893d6e9b343e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 43025 } }
I20260812 03:15:05.368259 4999 catalog_manager.cc:5495] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c28788214e478fb374421aeb11acdd (127.16.217.66). New cstate: current_term: 1 leader_uuid: "f3c28788214e478fb374421aeb11acdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1174bf486d496e83d70f723e38cdaf" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c28788214e478fb374421aeb11acdd" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c4b4b2d2d8495680bc893d6e9b343e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 43025 } health_report { overall_health: UNKNOWN } } }
I20260812 03:15:05.369432 5876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:05.345572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:51339 (local address 127.16.217.65:43025)
0812 03:15:05.345731 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:05.345736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:05.354871 (+ 9135us) server_negotiation.cc:407] Connection header received
0812 03:15:05.355210 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:05.355214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:05.355268 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:05.355318 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:15:05.355699 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.356465 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.368110 (+ 11645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.368732 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.368833 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:05.368886 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:05.369221 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 03:15:05.369305 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 03:15:05.369836 5899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:05.345032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:43025 (local address 127.16.217.66:51339)
0812 03:15:05.354788 (+ 9756us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:05.354804 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:05.355047 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:05.355458 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:05.355461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:05.355481 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:15:05.355571 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:05.355576 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.366715 (+ 11139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.366718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:05.367938 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:05.367951 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.368982 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.368985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:05.369069 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:05.369092 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:15:05.369630 (+ 538us) client_negotiation.cc:240] Negotiation successful
0812 03:15:05.369713 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9694,"thread_start_us":115,"threads_started":1}
I20260812 03:15:05.371409 5291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a46d3c0595e84950bf4fea30a1e1d30f" candidate_uuid: "f3c28788214e478fb374421aeb11acdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c4b4b2d2d8495680bc893d6e9b343e" is_pre_election: true
I20260812 03:15:05.371785 5291 raft_consensus.cc:2454] T a46d3c0595e84950bf4fea30a1e1d30f P 38c4b4b2d2d8495680bc893d6e9b343e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c28788214e478fb374421aeb11acdd in term 0.
I20260812 03:15:05.372241 5290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a46d3c0595e84950bf4fea30a1e1d30f" candidate_uuid: "f3c28788214e478fb374421aeb11acdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c4b4b2d2d8495680bc893d6e9b343e"
I20260812 03:15:05.372407 5290 raft_consensus.cc:3037] T a46d3c0595e84950bf4fea30a1e1d30f P 38c4b4b2d2d8495680bc893d6e9b343e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:15:05.373816 5290 raft_consensus.cc:2454] T a46d3c0595e84950bf4fea30a1e1d30f P 38c4b4b2d2d8495680bc893d6e9b343e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c28788214e478fb374421aeb11acdd in term 1.
I20260812 03:15:05.467614 5874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:05.462052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56406 (local address 127.16.217.66:35255)
0812 03:15:05.462278 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:05.462284 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:05.462305 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:15:05.462413 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:05.462417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:05.462499 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:05.462631 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:15:05.463083 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.463885 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.465186 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:05.465516 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:05.465626 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:05.466104 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:15:05.466749 (+ 645us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:15:05.466872 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:05.467382 (+ 510us) server_negotiation.cc:299] Negotiation successful
0812 03:15:05.467453 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 03:15:05.479717 5757 raft_consensus.cc:1261] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf [term 1 FOLLOWER]: Refusing update from remote peer f3c28788214e478fb374421aeb11acdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:15:05.479717 5290 raft_consensus.cc:1261] T a46d3c0595e84950bf4fea30a1e1d30f P 38c4b4b2d2d8495680bc893d6e9b343e [term 1 FOLLOWER]: Refusing update from remote peer f3c28788214e478fb374421aeb11acdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:15:05.480520 5900 consensus_queue.cc:1035] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c4b4b2d2d8495680bc893d6e9b343e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 43025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:15:05.481021 5900 consensus_queue.cc:1035] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae1174bf486d496e83d70f723e38cdaf" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 34589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:15:05.491778 5908 mvcc.cc:205] Tried to move back new op lower bound from 7317522454436503552 to 7317522453977079808. Current Snapshot: MvccSnapshot[applied={T|T < 7317522454436503552}]
I20260812 03:15:05.500535 5909 mvcc.cc:205] Tried to move back new op lower bound from 7317522454436503552 to 7317522453977079808. Current Snapshot: MvccSnapshot[applied={T|T < 7317522454436503552}]
I20260812 03:15:05.516587 5003 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:15:05.516804 5003 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:15:05.520265 5003 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ce1181053e54f25b3e089811740dd46: Sending DeleteTablet for 3 replicas of tablet a46d3c0595e84950bf4fea30a1e1d30f
I20260812 03:15:05.521178 5692 tablet_service.cc:1514] Processing DeleteTablet for tablet a46d3c0595e84950bf4fea30a1e1d30f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:15:05 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:60638
I20260812 03:15:05.522342 5500 tablet_service.cc:1514] Processing DeleteTablet for tablet a46d3c0595e84950bf4fea30a1e1d30f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:15:05 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:56392
I20260812 03:15:05.523330 5227 tablet_service.cc:1514] Processing DeleteTablet for tablet a46d3c0595e84950bf4fea30a1e1d30f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:15:05 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:36922
I20260812 03:15:05.524858 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 5154
I20260812 03:15:05.530400 5911 mvcc.cc:205] Tried to move back new op lower bound from 7317522454436503552 to 7317522453977079808. Current Snapshot: MvccSnapshot[applied={T|T < 7317522454436503552}]
I20260812 03:15:05.534514 5926 tablet_replica.cc:330] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf: stopping tablet replica
I20260812 03:15:05.535074 5927 tablet_replica.cc:330] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd: stopping tablet replica
I20260812 03:15:05.536696 5927 raft_consensus.cc:2229] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:15:05.542644 5926 raft_consensus.cc:2229] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:15:05.543063 5926 raft_consensus.cc:2258] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:15:05.546252 5927 raft_consensus.cc:2258] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:15:05.548033 5927 ts_tablet_manager.cc:1905] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:15:05.551877 5927 ts_tablet_manager.cc:1918] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:15:05.552000 5927 log.cc:1196] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/a46d3c0595e84950bf4fea30a1e1d30f
I20260812 03:15:05.552356 5927 ts_tablet_manager.cc:1939] T a46d3c0595e84950bf4fea30a1e1d30f P f3c28788214e478fb374421aeb11acdd: Deleting consensus metadata
W20260812 03:15:05.552929 4972 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.217.65:43025 (error 108)
I20260812 03:15:05.553750 4972 catalog_manager.cc:4841] TS f3c28788214e478fb374421aeb11acdd (127.16.217.66:35255): tablet a46d3c0595e84950bf4fea30a1e1d30f (table test-table [id=64e4b18d0e5d4e8b8349108a30deac97]) successfully deleted
I20260812 03:15:05.553830 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 5392
I20260812 03:15:05.565387 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:05.564441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:43025 (local address 127.0.0.1:36928)
0812 03:15:05.565119 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:05.565237 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:43025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":70,"threads_started":1}
W20260812 03:15:05.565567 4972 catalog_manager.cc:4584] TS 38c4b4b2d2d8495680bc893d6e9b343e (127.16.217.65:43025): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a46d3c0595e84950bf4fea30a1e1d30f: Network error: Client connection negotiation failed: client connection to 127.16.217.65:43025: connect: Connection refused (error 111)
I20260812 03:15:05.565858 5926 ts_tablet_manager.cc:1905] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:15:05.568960 5926 ts_tablet_manager.cc:1918] T a46d3c0595e84950bf4fea30a1e1d30f P ae1174bf486d496e83d70f723e38cdaf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:15:05.573817 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 5621
I20260812 03:15:05.592712 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 4949
I20260812 03:15:05.613297 17253 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 03:15:05.961218 17253 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 03:15:05.961387 17253 mini_ranger.cc:66] Stopping Ranger...
I20260812 03:15:06.317333 17253 mini_ranger.cc:68] Stopped Ranger
2026-08-12 03:15:06.317 UTC [4210] LOG: received smart shutdown request
2026-08-12 03:15:06.322 UTC [4210] LOG: background worker "logical replication launcher" (PID 4231) exited with exit code 1
2026-08-12 03:15:06.323 UTC [4216] LOG: shutting down
2026-08-12 03:15:06.486 UTC [4210] LOG: database system is shut down
2026-08-12T03:15:06Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (54709 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:15:06 dist-test-slave-7nm7 krb5kdc[5963](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:15:06 dist-test-slave-7nm7 krb5kdc[5963](info): set up 2 sockets
Aug 12 03:15:06 dist-test-slave-7nm7 krb5kdc[5963](info): commencing operation
krb5kdc: starting...
W20260812 03:15:07.959411 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:15:08 dist-test-slave-7nm7 krb5kdc[5963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:15:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:15:08Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.16.217.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.16.217.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.16.217.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/rangeradmin_127.16.217.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.16.217.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/rangeradmin_127.16.217.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.16.217.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.16.217.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.16.217.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/rangerlookup_127.16.217.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.16.217.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/rangerlookup_127.16.217.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.217.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.16.217.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.217.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/HTTP_127.16.217.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.217.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/HTTP_127.16.217.84@KRBTEST.COM.keytab.
I20260812 03:15:08.150270 17253 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskm8e4iX/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/postgres -l logfile start
2026-08-12 03:15:10.778 UTC [6013] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 03:15:10.778 UTC [6013] LOG: listening on IPv4 address "127.16.217.84", port 58043
2026-08-12 03:15:10.783 UTC [6013] LOG: listening on Unix socket "/tmp/.s.PGSQL.58043"
2026-08-12 03:15:10.821 UTC [6024] LOG: database system was shut down at 2026-08-12 03:15:09 UTC
2026-08-12 03:15:10.826 UTC [6013] LOG: database system is ready to accept connections
I20260812 03:15:12.046568 17253 mini_postgres.cc:96] Postgres bound to 58043
2026-08-12 03:15:12.054 UTC [6039] FATAL: database "slave" does not exist
127.16.217.84:58043 - accepting connections
I20260812 03:15:12.055160 17253 mini_ranger.cc:161] Starting Ranger...
I20260812 03:15:12.073285 17253 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 03:15:12.584530 17253 mini_ranger.cc:87] Created ranger Postgres database
I20260812 03:15:12.584632 17253 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-admin
2026-08-12 03:15:12,880 [I] DB FLAVOR :POSTGRES
2026-08-12 03:15:12,881 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:15:12,881 [I] Checking connection..
2026-08-12 03:15:12,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 03:15:13,132 [I] Checking connection passed.
2026-08-12 03:15:13,133 [I] --------- Verifying version history table ---------
2026-08-12 03:15:13,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:15:13,415 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:15:13,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:15:13,679 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:15:13,679 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 03:15:13,679 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 03:15:13.937 UTC [6143] WARNING: there is no transaction in progress
2026-08-12 03:15:13,947 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 03:15:13,947 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:15:14,213 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 03:15:14,213 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:15:14,213 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 03:15:14,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 03:15:14,765 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7nm7.c.gcp-upstream.internal','N') ;"
2026-08-12 03:15:15,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:15:15,280 [I] Table x_portal_user does not exist in database ranger
2026-08-12 03:15:15,281 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:15:15,530 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 03:15:15,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:15:15,771 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:15:16,028 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 03:15:16,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 03:15:17.090 UTC [6335] WARNING: there is no transaction in progress
2026-08-12 03:15:17.112 UTC [6335] WARNING: there is no transaction in progress
2026-08-12 03:15:17.130 UTC [6335] WARNING: there is no transaction in progress
2026-08-12 03:15:17.147 UTC [6335] WARNING: there is no transaction in progress
2026-08-12 03:15:17.164 UTC [6335] WARNING: there is no transaction in progress
2026-08-12 03:15:17.184 UTC [6335] WARNING: there is no transaction in progress
2026-08-12 03:15:17.253 UTC [6335] WARNING: there is no transaction in progress
2026-08-12 03:15:17.262 UTC [6335] WARNING: there is no transaction in progress
2026-08-12 03:15:17.278 UTC [6335] WARNING: there is no transaction in progress
2026-08-12 03:15:17.290 UTC [6335] WARNING: there is no transaction in progress
2026-08-12 03:15:17.302 UTC [6335] WARNING: there is no transaction in progress
2026-08-12 03:15:17.317 UTC [6335] WARNING: there is no transaction in progress
2026-08-12 03:15:17.327 UTC [6335] WARNING: there is no transaction in progress
2026-08-12 03:15:17.335 UTC [6335] WARNING: there is no transaction in progress
2026-08-12 03:15:17.342 UTC [6335] WARNING: there is no transaction in progress
2026-08-12 03:15:17.350 UTC [6335] WARNING: there is no transaction in progress
2026-08-12 03:15:18,054 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 03:15:18,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 03:15:18,324 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 03:15:18,325 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:15:18,610 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 03:15:18,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:15:18,883 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 03:15:18,884 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:15:19,136 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:15:19,393 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7nm7.c.gcp-upstream.internal';"
2026-08-12 03:15:19,653 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 03:15:19,654 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 03:15:19,906 [I] DB_PATCHES have already been applied
I20260812 03:15:19.912235 17253 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-admin:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskm8e4iX/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 03:15:19.912302 17253 mini_ranger.cc:193] Using host: 127.16.217.84
I20260812 03:15:19.917032 17253 mini_ranger.cc:227] Ranger admin URL: http://127.16.217.84:50825
Aug 12, 2026 3:15:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:15:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:15:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 3:15:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 3:15:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 3:15:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 3:15:20 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 03:15:21.399518 17253 mini_ranger.cc:160] Time spent starting Ranger: real 9.344s user 0.000s sys 0.013s
Aug 12, 2026 3:15:43 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 3:15:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 12, 2026 3:15:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 3:15:43 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 3:15:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 3:15:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 3:15:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 3:15:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 3:15:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 3:15:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 3:15:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 3:15:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 3:15:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 3:15:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 3:15:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 3:15:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 3:15:43 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 3:15:44 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 03:15:45.209156 17253 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.16.217.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.16.217.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.16.217.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/rangerkms_127.16.217.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.16.217.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/rangerkms_127.16.217.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.217.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.16.217.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.217.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/HTTP_127.16.217.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.217.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/HTTP_127.16.217.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 03:15:45 dist-test-slave-7nm7 krb5kdc[5963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504545, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 03:15:45.317842 17253 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 03:15:45.328735 17253 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 03:15:46.280023 17253 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 03:15:46.280140 17253 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-kms
I20260812 03:15:46.280166 17253 mini_ranger_kms.cc:227] Using postgres at 127.16.217.84:58043
2026-08-12 03:15:46,475 [I] DB FLAVOR :POSTGRES
2026-08-12 03:15:46,475 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:15:46,475 [I] Checking connection
2026-08-12 03:15:46,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 03:15:46,770 [I] connection success
2026-08-12 03:15:46,771 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 03:15:46,771 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 03:15:46,771 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 03:15:47,060 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 03:15:47,060 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:15:47,060 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 03:15:47,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.217.84:58043/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 03:15:47,370 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 03:15:47.852622 17253 mini_ranger_kms.cc:310] Created kms service
I20260812 03:15:48.044632 17253 mini_ranger_kms.cc:326] Created kudu user
I20260812 03:15:48.090685 17253 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 03:15:48 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504545, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
I20260812 03:15:48.532757 17253 mini_ranger_kms.cc:384] Added ranger policy
I20260812 03:15:48.532929 17253 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-kms:/tmp/dist-test-taskm8e4iX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskm8e4iX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 03:15:48.532956 17253 mini_ranger_kms.cc:240] Using host: 127.16.217.84
I20260812 03:15:48.538043 17253 mini_ranger_kms.cc:276] Ranger KMS PID: 6677
I20260812 03:15:48.538144 17253 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.16.217.84:50189
Aug 12, 2026 3:15:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:15:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:15:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 3:15:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 3:15:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 3:15:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 03:15:50 dist-test-slave-7nm7 krb5kdc[5963](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786504550, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.217.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 03:15:51 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504550, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.217.84@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
I20260812 03:15:51.591204 17253 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.273s user 0.008s sys 0.168s
I20260812 03:15:51.591353 17253 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 03:15:51.591439 17253 mini_ranger_kms.cc:401] 127.16.217.84:50189/kms/v1/keys
Aug 12 03:15:51 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504550, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.217.84@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
Aug 12 03:15:54 dist-test-slave-7nm7 krb5kdc[5963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:15:54.389586 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:36413
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:36413
--ranger_config_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.217.84:50189/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:15:54.540109 6789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:15:54.540393 6789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:15:54.540436 6789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:15:54.545210 6789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:15:54.545397 6789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:15:54.545472 6789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:15:54.545516 6789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:15:54.545552 6789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:15:54.551509 6789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/wal
--ranger_kms_url=127.16.217.84:50189/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:36413
--ranger_config_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:36413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:15:54.552670 6789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:15:54.553752 6789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:15:54.563308 6795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:15:54.563433 6789 server_base.cc:1034] running on GCE node
W20260812 03:15:54.563310 6796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:54.563452 6802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:15:54.564301 6789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:15:54.565460 6789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:15:54.566663 6789 hybrid_clock.cc:648] HybridClock initialized: now 1786504554566650 us; error 34 us; skew 500 ppm
Aug 12 03:15:54 dist-test-slave-7nm7 krb5kdc[5963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504554, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:15:54.574528 6789 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:15:54.575747 6789 webserver.cc:460] Webserver started at http://127.16.217.126:40919/ using document root <none> and password file <none>
I20260812 03:15:54.576576 6789 fs_manager.cc:359] Metadata directory not provided
I20260812 03:15:54.576686 6789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:15:54.576938 6789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:15:54 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504554, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
I20260812 03:15:54.629498 6789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "410521a341c04193b3640564c2439a50"
format_stamp: "Formatted at 2026-08-12 03:15:54 on dist-test-slave-7nm7"
server_key: "e8276f5360b8090b11cb72cbfe0d7b37"
server_key_iv: "5455888312352cd71c0406f29a8324f5"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:54.630182 6789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "410521a341c04193b3640564c2439a50"
format_stamp: "Formatted at 2026-08-12 03:15:54 on dist-test-slave-7nm7"
server_key: "e8276f5360b8090b11cb72cbfe0d7b37"
server_key_iv: "5455888312352cd71c0406f29a8324f5"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:54.634548 6789 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:15:54.653967 6819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:54.655944 6789 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:15:54.656122 6789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "410521a341c04193b3640564c2439a50"
format_stamp: "Formatted at 2026-08-12 03:15:54 on dist-test-slave-7nm7"
server_key: "e8276f5360b8090b11cb72cbfe0d7b37"
server_key_iv: "5455888312352cd71c0406f29a8324f5"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:54.656296 6789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:15:54.673938 6789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:15:54.677017 6789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:15:54.677254 6789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:15:54.685531 6789 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:36413
I20260812 03:15:54.685709 6917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:36413 every 8 connection(s)
I20260812 03:15:54.686836 6789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:15:54.689878 6918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:15:54.690727 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 6789
I20260812 03:15:54.690843 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:15:54.691179 17253 external_mini_cluster.cc:1428] Setting key c20d45794a9223213be158e1d427511d
I20260812 03:15:54.697319 6918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50: Bootstrap starting.
I20260812 03:15:54.699981 6918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50: Neither blocks nor log segments found. Creating new log.
Aug 12 03:15:54 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:15:54.700842 6918 log.cc:824] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50: Log is configured to *not* fsync() on all Append() calls
I20260812 03:15:54.703326 6918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50: No bootstrap required, opened a new log
I20260812 03:15:54.706525 6921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:54.692709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43712 (local address 127.16.217.126:36413)
0812 03:15:54.693171 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:54.693186 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:54.693231 (+ 45us) server_negotiation.cc:407] Connection header received
0812 03:15:54.693949 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:54.693967 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:54.694394 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:54.694744 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:15:54.695588 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:54.696582 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:54.697484 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:54.697967 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:54.698145 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:54.700841 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:15:54.700846 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:15:54.700862 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:15:54.700890 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:15:54.703343 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:54.703834 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:54.703840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:54.703845 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:54.703953 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:54.704248 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:54.704252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:54.704254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:54.704665 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:15:54.704908 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:54.705321 (+ 413us) server_negotiation.cc:299] Negotiation successful
0812 03:15:54.705598 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":143,"threads_started":1}
I20260812 03:15:54.706960 6918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "410521a341c04193b3640564c2439a50" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36413 } }
I20260812 03:15:54.707258 6918 raft_consensus.cc:374] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:15:54.707319 6918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 410521a341c04193b3640564c2439a50, State: Initialized, Role: FOLLOWER
I20260812 03:15:54.707782 6918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "410521a341c04193b3640564c2439a50" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36413 } }
I20260812 03:15:54.707916 6918 raft_consensus.cc:388] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:15:54.708000 6918 raft_consensus.cc:482] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:15:54.708107 6918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:15:54.709378 6918 raft_consensus.cc:504] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "410521a341c04193b3640564c2439a50" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36413 } }
I20260812 03:15:54.709774 6918 leader_election.cc:302] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 410521a341c04193b3640564c2439a50; no voters:
I20260812 03:15:54.710186 6918 leader_election.cc:288] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:15:54.710291 6923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:15:54.710898 6923 raft_consensus.cc:686] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [term 1 LEADER]: Becoming Leader. State: Replica: 410521a341c04193b3640564c2439a50, State: Running, Role: LEADER
I20260812 03:15:54.711266 6923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "410521a341c04193b3640564c2439a50" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36413 } }
I20260812 03:15:54.711699 6918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:15:54.713052 6927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 410521a341c04193b3640564c2439a50. Latest consensus state: current_term: 1 leader_uuid: "410521a341c04193b3640564c2439a50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "410521a341c04193b3640564c2439a50" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36413 } } }
I20260812 03:15:54.713214 6927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [sys.catalog]: This master's current role is: LEADER
I20260812 03:15:54.713678 6924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "410521a341c04193b3640564c2439a50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "410521a341c04193b3640564c2439a50" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36413 } } }
I20260812 03:15:54.713769 6924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [sys.catalog]: This master's current role is: LEADER
I20260812 03:15:54.714145 6918 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 03:15:54.714617 6941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:15:54.721186 6941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:15:54.728737 6941 catalog_manager.cc:1347] Generated new cluster ID: f2d28c0b7ae340c685b97061302d2d38
I20260812 03:15:54.728828 6941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:15:54.753440 6941 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:15:54.754895 6941 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:15:54.766458 6941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50: Generated new TSK 0
I20260812 03:15:54.767400 6941 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
I20260812 03:15:55.489164 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.217.84:50189/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:36413
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:15:55.626539 6997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:15:55.626945 6997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:15:55.627069 6997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:15:55.631798 6997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:15:55.631889 6997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:15:55.631984 6997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:15:55.637993 6997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.16.217.84:50189/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:36413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:15:55.639427 6997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:15:55.640558 6997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:15:55.653301 7007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:55.653306 7005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:15:55.654668 6997 server_base.cc:1034] running on GCE node
W20260812 03:15:55.654924 7009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:15:55.655673 6997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:15:55.656587 6997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:15:55.657816 6997 hybrid_clock.cc:648] HybridClock initialized: now 1786504555657745 us; error 87 us; skew 500 ppm
Aug 12 03:15:55 dist-test-slave-7nm7 krb5kdc[5963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504555, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:15:55.661744 6997 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:15:55.663218 6997 webserver.cc:460] Webserver started at http://127.16.217.65:35619/ using document root <none> and password file <none>
I20260812 03:15:55.664022 6997 fs_manager.cc:359] Metadata directory not provided
I20260812 03:15:55.664122 6997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:15:55.664381 6997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:15:55 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504555, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
I20260812 03:15:55.683082 6997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "d47b6b336aca4e59b5b41b4b273a6623"
format_stamp: "Formatted at 2026-08-12 03:15:55 on dist-test-slave-7nm7"
server_key: "84cda9db84c59edcafbe932c985a3166"
server_key_iv: "01356da76e85909d062eb452be8693ab"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:55.684024 6997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "d47b6b336aca4e59b5b41b4b273a6623"
format_stamp: "Formatted at 2026-08-12 03:15:55 on dist-test-slave-7nm7"
server_key: "84cda9db84c59edcafbe932c985a3166"
server_key_iv: "01356da76e85909d062eb452be8693ab"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:55.691882 6997 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s
I20260812 03:15:55.714150 7020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:55.719065 6997 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.004s sys 0.000s
I20260812 03:15:55.719331 6997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "d47b6b336aca4e59b5b41b4b273a6623"
format_stamp: "Formatted at 2026-08-12 03:15:55 on dist-test-slave-7nm7"
server_key: "84cda9db84c59edcafbe932c985a3166"
server_key_iv: "01356da76e85909d062eb452be8693ab"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:55.719575 6997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:15:55.754804 6997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:15:55.759352 6997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:15:55.759768 6997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:15:55.761083 6997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:15:55.762399 6997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:15:55.762516 6997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:55.762701 6997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:15:55.762787 6997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:55.780953 6997 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:42301
I20260812 03:15:55.782797 6997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:15:55.783793 7223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:42301 every 8 connection(s)
I20260812 03:15:55.789647 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 6997
I20260812 03:15:55.789785 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:15:55.790093 17253 external_mini_cluster.cc:1428] Setting key aee783f1aeefb4f68594b906b2701b4c
Aug 12 03:15:55 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504555, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:15:55.806414 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:55.786213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:33817 (local address 127.16.217.126:36413)
0812 03:15:55.786854 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:55.786860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:55.787533 (+ 673us) server_negotiation.cc:407] Connection header received
0812 03:15:55.788679 (+ 1146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:55.788683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:55.788742 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:55.788845 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:15:55.791846 (+ 3001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:55.792717 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:55.794680 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:55.795059 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:55.795145 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:55.801017 (+ 5872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:15:55.801022 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:15:55.801025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:15:55.801068 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:15:55.803207 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:55.804061 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:55.804067 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:55.804071 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:55.804182 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:55.804685 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:55.804703 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:55.804728 (+ 25us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:55.805003 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:15:55.805101 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:55.805958 (+ 857us) server_negotiation.cc:299] Negotiation successful
0812 03:15:55.806136 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.16.217.66@KRBTEST.COM; defaulting to no policy
I20260812 03:15:55.808296 7237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:55.786735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:36413 (local address 127.16.217.65:33817)
0812 03:15:55.787351 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:55.787414 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:55.788381 (+ 967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:55.789066 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:55.789075 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:55.790857 (+ 1782us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:15:55.791646 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:55.791672 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:55.792900 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:55.792903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:55.794522 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:55.794535 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:55.795440 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:55.795444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:55.795649 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:55.796424 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:15:55.796455 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:15:55.798428 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:15:55.803392 (+ 4964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:15:55.803403 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:15:55.803428 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:15:55.803910 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:15:55.804325 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:15:55.804330 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:15:55.804335 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:15:55.804554 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:15:55.805188 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:15:55.805196 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:15:55.805610 (+ 414us) client_negotiation.cc:769] Sending connection context
0812 03:15:55.806767 (+ 1157us) client_negotiation.cc:240] Negotiation successful
0812 03:15:55.807191 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":153,"threads_started":1}
I20260812 03:15:55.810061 7229 heartbeater.cc:347] Connected to a master server at 127.16.217.126:36413
I20260812 03:15:55.810323 7229 heartbeater.cc:464] Registering TS with master...
I20260812 03:15:55.810938 7229 heartbeater.cc:511] Master 127.16.217.126:36413 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.66@KRBTEST.COM" created.
I20260812 03:15:55.812956 6853 ts_manager.cc:194] Registered new tserver with Master: d47b6b336aca4e59b5b41b4b273a6623 (127.16.217.65:42301)
I20260812 03:15:55.814183 6853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:33817
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
I20260812 03:15:55.834957 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.217.84:50189/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:36413
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:15:55 dist-test-slave-7nm7 krb5kdc[5963](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786504555, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 03:15:55.967937 7241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:15:55.968258 7241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:15:55.968364 7241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:15:55.972681 7241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:15:55.972783 7241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:15:55.972893 7241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:15:55.979653 7241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.16.217.84:50189/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:36413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:15:55.980876 7241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:15:55.982280 7241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:15:55.992595 7256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:55.992543 7254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:55.993638 7253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:15:55.993716 7241 server_base.cc:1034] running on GCE node
I20260812 03:15:55.994202 7241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:15:55.995157 7241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:15:55.996474 7241 hybrid_clock.cc:648] HybridClock initialized: now 1786504555996435 us; error 54 us; skew 500 ppm
Aug 12 03:15:55 dist-test-slave-7nm7 krb5kdc[5963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504555, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:15:56.000438 7241 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:15:56.002141 7241 webserver.cc:460] Webserver started at http://127.16.217.66:35103/ using document root <none> and password file <none>
I20260812 03:15:56.002897 7241 fs_manager.cc:359] Metadata directory not provided
I20260812 03:15:56.002998 7241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:15:56.003266 7241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:15:55 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504555, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
I20260812 03:15:56.023458 7241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "da7c473f215c476589a48af5aaa78ba3"
format_stamp: "Formatted at 2026-08-12 03:15:56 on dist-test-slave-7nm7"
server_key: "64c7523c360c24e8d4d695215860b419"
server_key_iv: "3052799c3156f9f528a8b818fd1416cf"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:56.024247 7241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "da7c473f215c476589a48af5aaa78ba3"
format_stamp: "Formatted at 2026-08-12 03:15:56 on dist-test-slave-7nm7"
server_key: "64c7523c360c24e8d4d695215860b419"
server_key_iv: "3052799c3156f9f528a8b818fd1416cf"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:56.029136 7241 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 03:15:56.050017 7267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:56.051858 7241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:15:56.052090 7241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "da7c473f215c476589a48af5aaa78ba3"
format_stamp: "Formatted at 2026-08-12 03:15:56 on dist-test-slave-7nm7"
server_key: "64c7523c360c24e8d4d695215860b419"
server_key_iv: "3052799c3156f9f528a8b818fd1416cf"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:56.052343 7241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:15:56.087992 7241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:15:56.091180 7241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:15:56.091394 7241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:15:56.092114 7241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:15:56.093066 7241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:15:56.093137 7241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:56.093219 7241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:15:56.093410 7241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:56.105482 7241 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:41933
I20260812 03:15:56.105587 7469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:41933 every 8 connection(s)
I20260812 03:15:56.106660 7241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:15:56.114604 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 7241
I20260812 03:15:56.114745 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:15:56.115079 17253 external_mini_cluster.cc:1428] Setting key 4eed78161c260ec2fefcbf0b724a9e33
Aug 12 03:15:56 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504555, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:15:56.125002 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:56.108668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:50187 (local address 127.16.217.126:36413)
0812 03:15:56.108820 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:56.108825 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:56.110605 (+ 1780us) server_negotiation.cc:407] Connection header received
0812 03:15:56.111956 (+ 1351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:56.111960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:56.112022 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:56.112081 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:15:56.113923 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:56.114757 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:56.116407 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:56.116746 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:56.116830 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:56.120311 (+ 3481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:15:56.120315 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:15:56.120319 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:15:56.120348 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:15:56.122283 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:56.122966 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:56.122970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:56.122972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:56.123060 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:56.123521 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:56.123525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:56.123527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:56.123770 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:15:56.123848 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:56.124644 (+ 796us) server_negotiation.cc:299] Negotiation successful
0812 03:15:56.124823 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 03:15:56.127177 7474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:56.109161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:36413 (local address 127.16.217.66:50187)
0812 03:15:56.110430 (+ 1269us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:56.110489 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:56.111681 (+ 1192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:56.112608 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:56.112617 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:56.112964 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:15:56.113740 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:56.113763 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:56.114941 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:56.114945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:56.116271 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:56.116282 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:56.117046 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:56.117050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:56.117276 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:56.118024 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:15:56.118051 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:15:56.120073 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:15:56.122442 (+ 2369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:15:56.122449 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:15:56.122468 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:15:56.122840 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:15:56.123173 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:15:56.123175 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:15:56.123178 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:15:56.123393 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:15:56.123937 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:15:56.123944 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:15:56.124326 (+ 382us) client_negotiation.cc:769] Sending connection context
0812 03:15:56.125567 (+ 1241us) client_negotiation.cc:240] Negotiation successful
0812 03:15:56.125951 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1004,"thread_start_us":158,"threads_started":1}
I20260812 03:15:56.128964 7471 heartbeater.cc:347] Connected to a master server at 127.16.217.126:36413
I20260812 03:15:56.129319 7471 heartbeater.cc:464] Registering TS with master...
I20260812 03:15:56.130038 7471 heartbeater.cc:511] Master 127.16.217.126:36413 requested a full tablet report, sending...
I20260812 03:15:56.131346 6853 ts_manager.cc:194] Registered new tserver with Master: da7c473f215c476589a48af5aaa78ba3 (127.16.217.66:41933)
I20260812 03:15:56.131960 6853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:50187
WARNING: no policy specified for kudu/127.16.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
I20260812 03:15:56.166179 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.217.84:50189/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:36413
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:15:56.316226 7477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:15:56.316525 7477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:15:56.316654 7477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:15:56.321060 7477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:15:56.321167 7477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:15:56.321274 7477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:15:56.327922 7477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.16.217.84:50189/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:36413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:15:56.329648 7477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:15:56.331076 7477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:15:56.348160 7493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:15:56.349440 7477 server_base.cc:1034] running on GCE node
W20260812 03:15:56.349802 7488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:56.350409 7485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:15:56.351054 7477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:15:56.351939 7477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:15:56.353166 7477 hybrid_clock.cc:648] HybridClock initialized: now 1786504556353102 us; error 83 us; skew 500 ppm
Aug 12 03:15:56 dist-test-slave-7nm7 krb5kdc[5963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504556, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:15:56.357100 7477 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:15:56.358403 7477 webserver.cc:460] Webserver started at http://127.16.217.67:38547/ using document root <none> and password file <none>
I20260812 03:15:56.359177 7477 fs_manager.cc:359] Metadata directory not provided
I20260812 03:15:56.359261 7477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:15:56.359505 7477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:15:56 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504556, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
Aug 12 03:15:56 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504555, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
I20260812 03:15:56.383554 7477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "f18883616ef14ec2bfc0d3636e5103ae"
format_stamp: "Formatted at 2026-08-12 03:15:56 on dist-test-slave-7nm7"
server_key: "b3c345eee4bb297e6877aa81c796fa7c"
server_key_iv: "12d5e165fa8483afd10a5eadec3a9f30"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:56.384371 7477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "f18883616ef14ec2bfc0d3636e5103ae"
format_stamp: "Formatted at 2026-08-12 03:15:56 on dist-test-slave-7nm7"
server_key: "b3c345eee4bb297e6877aa81c796fa7c"
server_key_iv: "12d5e165fa8483afd10a5eadec3a9f30"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:56.389751 7477 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 03:15:56.411221 7512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:56.413177 7477 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.001s
I20260812 03:15:56.413334 7477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "f18883616ef14ec2bfc0d3636e5103ae"
format_stamp: "Formatted at 2026-08-12 03:15:56 on dist-test-slave-7nm7"
server_key: "b3c345eee4bb297e6877aa81c796fa7c"
server_key_iv: "12d5e165fa8483afd10a5eadec3a9f30"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:56.413487 7477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:15:56.432644 7477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:15:56.436691 7477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:15:56.436936 7477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:15:56.437747 7477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:15:56.439035 7477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:15:56.439113 7477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:56.439196 7477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:15:56.439236 7477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:56.452914 7477 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:39603
I20260812 03:15:56.452980 7719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:39603 every 8 connection(s)
I20260812 03:15:56.454056 7477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:15:56.459223 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 7477
I20260812 03:15:56.459350 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:15:56.459635 17253 external_mini_cluster.cc:1428] Setting key 99e96fc4ce910354425d80abedbcd056
Aug 12 03:15:56 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504556, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:15:56.473824 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:56.458223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:44083 (local address 127.16.217.126:36413)
0812 03:15:56.458418 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:56.458423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:56.460473 (+ 2050us) server_negotiation.cc:407] Connection header received
0812 03:15:56.461846 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:56.461850 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:56.461901 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:56.461947 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:15:56.463748 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:56.464574 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:56.466200 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:56.466589 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:56.466713 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:56.469806 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:15:56.469810 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:15:56.469813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:15:56.469843 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:15:56.471599 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:56.472174 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:56.472179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:56.472181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:56.472276 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:56.472649 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:56.472654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:56.472656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:56.472856 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:15:56.472975 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:56.473517 (+ 542us) server_negotiation.cc:299] Negotiation successful
0812 03:15:56.473648 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:15:56.474849 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:56.458793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:36413 (local address 127.16.217.67:44083)
0812 03:15:56.460292 (+ 1499us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:56.460354 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:56.461568 (+ 1214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:56.462468 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:56.462477 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:56.462899 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:15:56.463577 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:56.463596 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:56.464730 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:56.464733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:56.466045 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:56.466057 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:56.466762 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:56.466766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:56.466969 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:56.467719 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:15:56.467740 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:15:56.469595 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:15:56.471750 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:15:56.471757 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:15:56.471776 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:15:56.472056 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:15:56.472378 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:15:56.472382 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:15:56.472384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:15:56.472545 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:15:56.472973 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:15:56.472981 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:15:56.473268 (+ 287us) client_negotiation.cc:769] Sending connection context
0812 03:15:56.473476 (+ 208us) client_negotiation.cc:240] Negotiation successful
0812 03:15:56.473832 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1248,"thread_start_us":149,"threads_started":1}
I20260812 03:15:56.476109 7720 heartbeater.cc:347] Connected to a master server at 127.16.217.126:36413
I20260812 03:15:56.476402 7720 heartbeater.cc:464] Registering TS with master...
I20260812 03:15:56.476985 7720 heartbeater.cc:511] Master 127.16.217.126:36413 requested a full tablet report, sending...
I20260812 03:15:56.478140 6853 ts_manager.cc:194] Registered new tserver with Master: f18883616ef14ec2bfc0d3636e5103ae (127.16.217.67:39603)
I20260812 03:15:56.478950 6853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:44083
I20260812 03:15:56.479142 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:15:56.479940 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 6997
I20260812 03:15:56.501731 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 7241
I20260812 03:15:56.517459 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 7477
I20260812 03:15:56.544196 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 6789
I20260812 03:15:56.557966 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:36413
--webserver_interface=127.16.217.126
--webserver_port=40919
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:36413
--ranger_config_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.217.84:50189/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.217.84:50189/kms with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:15:56.680877 7730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:15:56.681167 7730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:15:56.681216 7730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:15:56.685490 7730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:15:56.685570 7730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:15:56.685590 7730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:15:56.685606 7730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:15:56.685621 7730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:15:56.691502 7730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.217.84:50189/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:36413
--ranger_config_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:36413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=40919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:15:56.692772 7730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:15:56.693900 7730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:15:56.703087 7745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:56.703087 7738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:56.703119 7737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:15:56.703641 7730 server_base.cc:1034] running on GCE node
I20260812 03:15:56.704442 7730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:15:56.705816 7730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:15:56.707011 7730 hybrid_clock.cc:648] HybridClock initialized: now 1786504556706930 us; error 94 us; skew 500 ppm
Aug 12 03:15:56 dist-test-slave-7nm7 krb5kdc[5963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504556, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:15:56.710544 7730 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:15:56.711618 7730 webserver.cc:460] Webserver started at http://127.16.217.126:40919/ using document root <none> and password file <none>
I20260812 03:15:56.712232 7730 fs_manager.cc:359] Metadata directory not provided
I20260812 03:15:56.712342 7730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:15:56 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504556, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
I20260812 03:15:56.760862 7730 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:15:56.763938 7758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:56.765601 7730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:15:56.765750 7730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "410521a341c04193b3640564c2439a50"
format_stamp: "Formatted at 2026-08-12 03:15:54 on dist-test-slave-7nm7"
server_key: "e8276f5360b8090b11cb72cbfe0d7b37"
server_key_iv: "5455888312352cd71c0406f29a8324f5"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:56.766522 7730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:15:56.784848 7730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:15:56.788676 7730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:15:56.788890 7730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:15:56.798074 7855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:36413 every 8 connection(s)
I20260812 03:15:56.798076 7730 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:36413
I20260812 03:15:56.799552 7730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:15:56.804224 7856 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:15:56.805900 7856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50: Bootstrap starting.
I20260812 03:15:56.807590 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 7730
I20260812 03:15:56.808110 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.217.84:50189/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:42301
--local_ip_for_outbound_sockets=127.16.217.65
--tserver_master_addrs=127.16.217.126:36413
--webserver_port=35619
--webserver_interface=127.16.217.65
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.217.84:50189/kms with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:15:56.823845 7856 log.cc:824] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50: Log is configured to *not* fsync() on all Append() calls
I20260812 03:15:56.830848 7856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:15:56.831300 7856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50: Bootstrap complete.
I20260812 03:15:56.835825 7856 raft_consensus.cc:348] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "410521a341c04193b3640564c2439a50" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36413 } }
I20260812 03:15:56.836310 7856 raft_consensus.cc:729] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 410521a341c04193b3640564c2439a50, State: Initialized, Role: FOLLOWER
I20260812 03:15:56.836874 7856 consensus_queue.cc:260] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "410521a341c04193b3640564c2439a50" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36413 } }
I20260812 03:15:56.837026 7856 raft_consensus.cc:388] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:15:56.837092 7856 raft_consensus.cc:482] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:15:56.837200 7856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:15:56.839248 7856 raft_consensus.cc:504] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "410521a341c04193b3640564c2439a50" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36413 } }
I20260812 03:15:56.839596 7856 leader_election.cc:302] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 410521a341c04193b3640564c2439a50; no voters:
I20260812 03:15:56.839986 7856 leader_election.cc:288] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:15:56.840147 7861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:15:56.840432 7861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [term 2 LEADER]: Becoming Leader. State: Replica: 410521a341c04193b3640564c2439a50, State: Running, Role: LEADER
I20260812 03:15:56.840772 7861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "410521a341c04193b3640564c2439a50" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36413 } }
I20260812 03:15:56.841320 7856 sys_catalog.cc:564] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:15:56.841861 7861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "410521a341c04193b3640564c2439a50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "410521a341c04193b3640564c2439a50" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36413 } } }
I20260812 03:15:56.842046 7861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [sys.catalog]: This master's current role is: LEADER
I20260812 03:15:56.842357 7861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 410521a341c04193b3640564c2439a50. Latest consensus state: current_term: 2 leader_uuid: "410521a341c04193b3640564c2439a50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "410521a341c04193b3640564c2439a50" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36413 } } }
I20260812 03:15:56.842518 7861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50 [sys.catalog]: This master's current role is: LEADER
I20260812 03:15:56.842835 7872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:15:56.846479 7872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:15:56.847024 7856 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 03:15:56.847436 7872 catalog_manager.cc:1259] Loaded cluster ID: f2d28c0b7ae340c685b97061302d2d38
I20260812 03:15:56.847609 7872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:15:56.850029 7872 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:15:56.850843 7872 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50: Loaded TSK: 0
I20260812 03:15:56.852140 7872 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:15:56.979485 7858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:15:56.979971 7858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:15:56.980094 7858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:15:56.986433 7858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:15:56.986603 7858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:15:56.986768 7858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:15:56.994694 7858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.217.84:50189/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:42301
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=35619
--tserver_master_addrs=127.16.217.126:36413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:15:56.996270 7858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:15:56.997668 7858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:15:57.011504 7921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:57.011504 7916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:57.012104 7917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:15:57.013463 7858 server_base.cc:1034] running on GCE node
I20260812 03:15:57.014035 7858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:15:57.014973 7858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:15:57.031139 7858 hybrid_clock.cc:648] HybridClock initialized: now 1786504557031002 us; error 172 us; skew 500 ppm
Aug 12 03:15:57 dist-test-slave-7nm7 krb5kdc[5963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504557, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:15:57.036265 7858 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:15:57.037662 7858 webserver.cc:460] Webserver started at http://127.16.217.65:35619/ using document root <none> and password file <none>
I20260812 03:15:57.038698 7858 fs_manager.cc:359] Metadata directory not provided
I20260812 03:15:57.038877 7858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:15:57 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504557, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
I20260812 03:15:57.067780 7858 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.003s sys 0.001s
I20260812 03:15:57.071774 7935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:57.073638 7858 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:15:57.073812 7858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "d47b6b336aca4e59b5b41b4b273a6623"
format_stamp: "Formatted at 2026-08-12 03:15:55 on dist-test-slave-7nm7"
server_key: "84cda9db84c59edcafbe932c985a3166"
server_key_iv: "01356da76e85909d062eb452be8693ab"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:57.074692 7858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:15:57.107923 7858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:15:57.111848 7858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:15:57.112100 7858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:15:57.112910 7858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:15:57.114071 7858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:15:57.114137 7858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:57.114199 7858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:15:57.114245 7858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:57.127355 7858 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:42301
I20260812 03:15:57.128540 8132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:42301 every 8 connection(s)
I20260812 03:15:57.128832 7858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:15:57.133695 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 7858
I20260812 03:15:57.134187 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.217.84:50189/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:41933
--local_ip_for_outbound_sockets=127.16.217.66
--tserver_master_addrs=127.16.217.126:36413
--webserver_port=35103
--webserver_interface=127.16.217.66
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.217.84:50189/kms with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:15:57 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504557, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:15:57.159541 8136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:57.131866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:36413 (local address 127.16.217.65:35345)
0812 03:15:57.133121 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:57.133176 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:57.136163 (+ 2987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:57.138717 (+ 2554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:57.138728 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:57.139273 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:15:57.139966 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:57.139989 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:57.142224 (+ 2235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:57.142228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:57.143830 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:57.143843 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:57.149805 (+ 5962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:57.149810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:57.150035 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:57.150813 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:15:57.150836 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:15:57.152694 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:15:57.155522 (+ 2828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:15:57.155529 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:15:57.155547 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:15:57.155921 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:15:57.156386 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:15:57.156390 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:15:57.156392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:15:57.156582 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:15:57.157485 (+ 903us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:15:57.157492 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:15:57.157794 (+ 302us) client_negotiation.cc:769] Sending connection context
0812 03:15:57.158007 (+ 213us) client_negotiation.cc:240] Negotiation successful
0812 03:15:57.158410 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1025,"thread_start_us":157,"threads_started":1}
I20260812 03:15:57.161381 8138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:57.132176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:35345 (local address 127.16.217.126:36413)
0812 03:15:57.132797 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:57.132820 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:57.135102 (+ 2282us) server_negotiation.cc:407] Connection header received
0812 03:15:57.136826 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:57.136842 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:57.137546 (+ 704us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:57.137930 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:15:57.140225 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:57.141945 (+ 1720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:57.143989 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:57.144766 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:57.144940 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:57.152997 (+ 8057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:15:57.153005 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:15:57.153026 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:15:57.153066 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:15:57.155354 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:57.156081 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:57.156089 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:57.156095 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:57.156266 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:57.156701 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:57.156705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:57.156706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:57.157285 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:15:57.157424 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:57.160181 (+ 2757us) server_negotiation.cc:299] Negotiation successful
0812 03:15:57.160491 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":117,"threads_started":1}
I20260812 03:15:57.164155 8133 heartbeater.cc:347] Connected to a master server at 127.16.217.126:36413
I20260812 03:15:57.164427 8133 heartbeater.cc:464] Registering TS with master...
I20260812 03:15:57.165066 8133 heartbeater.cc:511] Master 127.16.217.126:36413 requested a full tablet report, sending...
W20260812 03:15:57.167855 8133 heartbeater.cc:644] Failed to heartbeat to 127.16.217.126:36413 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 03:15:57.167996 8133 heartbeater.cc:464] Registering TS with master...
I20260812 03:15:57.168226 8133 heartbeater.cc:511] Master 127.16.217.126:36413 requested a full tablet report, sending...
I20260812 03:15:57.169305 8133 heartbeater.cc:464] Registering TS with master...
I20260812 03:15:57.169490 8133 heartbeater.cc:511] Master 127.16.217.126:36413 requested a full tablet report, sending...
W20260812 03:15:57.170441 8133 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:15:57.316918 8140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:15:57.317220 8140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:15:57.317394 8140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:15:57.322378 8140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:15:57.322453 8140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:15:57.322554 8140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:15:57.329375 8140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.217.84:50189/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:41933
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=35103
--tserver_master_addrs=127.16.217.126:36413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:15:57.330868 8140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:15:57.332187 8140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:15:57.346143 8147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:57.346547 8149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:57.346119 8151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:15:57.346722 8140 server_base.cc:1034] running on GCE node
I20260812 03:15:57.347214 8140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:15:57.348059 8140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:15:57.349803 8140 hybrid_clock.cc:648] HybridClock initialized: now 1786504557349775 us; error 43 us; skew 500 ppm
Aug 12 03:15:57 dist-test-slave-7nm7 krb5kdc[5963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504557, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:15:57.353547 8140 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:15:57.357237 8140 webserver.cc:460] Webserver started at http://127.16.217.66:35103/ using document root <none> and password file <none>
I20260812 03:15:57.358106 8140 fs_manager.cc:359] Metadata directory not provided
I20260812 03:15:57.358309 8140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:15:57 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504557, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
I20260812 03:15:57.408123 8140 fs_manager.cc:709] Time spent opening directory manager: real 0.017s user 0.003s sys 0.000s
I20260812 03:15:57.411700 8163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:57.414152 8140 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260812 03:15:57.414436 8140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "da7c473f215c476589a48af5aaa78ba3"
format_stamp: "Formatted at 2026-08-12 03:15:56 on dist-test-slave-7nm7"
server_key: "64c7523c360c24e8d4d695215860b419"
server_key_iv: "3052799c3156f9f528a8b818fd1416cf"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:57.415370 8140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:15:57.436739 8140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:15:57.441170 8140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:15:57.441479 8140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:15:57.442466 8140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:15:57.443670 8140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:15:57.443783 8140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:57.443897 8140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:15:57.443969 8140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:57.458071 8140 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:41933
I20260812 03:15:57.458280 8342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:41933 every 8 connection(s)
I20260812 03:15:57.459388 8140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:15:57.460773 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 8140
I20260812 03:15:57.461310 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.217.84:50189/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:39603
--local_ip_for_outbound_sockets=127.16.217.67
--tserver_master_addrs=127.16.217.126:36413
--webserver_port=38547
--webserver_interface=127.16.217.67
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.217.84:50189/kms with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:15:57 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504557, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:15:57.480813 8138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:57.462886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:44917 (local address 127.16.217.126:36413)
0812 03:15:57.463074 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:57.463079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:57.465103 (+ 2024us) server_negotiation.cc:407] Connection header received
0812 03:15:57.466796 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:57.466803 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:57.466885 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:57.466962 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:15:57.468922 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:57.469857 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:57.471952 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:57.472451 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:57.472553 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:57.476303 (+ 3750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:15:57.476310 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:15:57.476314 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:15:57.476351 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:15:57.478258 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:57.478898 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:57.478902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:57.478904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:57.478992 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:57.479398 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:57.479401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:57.479403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:57.479631 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:15:57.479711 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:57.480497 (+ 786us) server_negotiation.cc:299] Negotiation successful
0812 03:15:57.480635 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:15:57.482784 8350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:57.463549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:36413 (local address 127.16.217.66:44917)
0812 03:15:57.464909 (+ 1360us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:57.464977 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:57.466494 (+ 1517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:57.467464 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:57.467479 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:57.467900 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:15:57.468703 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:57.468728 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:57.470055 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:57.470060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:57.471783 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:57.471801 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:57.472901 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:57.472909 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:57.473168 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:57.473991 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:15:57.474017 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:15:57.476034 (+ 2017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:15:57.478413 (+ 2379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:15:57.478420 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:15:57.478438 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:15:57.478780 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:15:57.479096 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:15:57.479099 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:15:57.479101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:15:57.479287 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:15:57.479780 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:15:57.479786 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:15:57.480190 (+ 404us) client_negotiation.cc:769] Sending connection context
0812 03:15:57.481349 (+ 1159us) client_negotiation.cc:240] Negotiation successful
0812 03:15:57.481699 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1087,"thread_start_us":159,"threads_started":1}
I20260812 03:15:57.484416 8343 heartbeater.cc:347] Connected to a master server at 127.16.217.126:36413
I20260812 03:15:57.484727 8343 heartbeater.cc:464] Registering TS with master...
I20260812 03:15:57.485352 8343 heartbeater.cc:511] Master 127.16.217.126:36413 requested a full tablet report, sending...
W20260812 03:15:57.487147 8343 heartbeater.cc:644] Failed to heartbeat to 127.16.217.126:36413 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 03:15:57.487298 8343 heartbeater.cc:464] Registering TS with master...
I20260812 03:15:57.487473 8343 heartbeater.cc:511] Master 127.16.217.126:36413 requested a full tablet report, sending...
I20260812 03:15:57.488333 8343 heartbeater.cc:464] Registering TS with master...
I20260812 03:15:57.488497 8343 heartbeater.cc:511] Master 127.16.217.126:36413 requested a full tablet report, sending...
W20260812 03:15:57.489306 8343 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:15:57.608745 8347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:15:57.609062 8347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:15:57.609159 8347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:15:57.614775 8347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:15:57.614907 8347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:15:57.615036 8347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:15:57.623176 8347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.217.84:50189/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:39603
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=38547
--tserver_master_addrs=127.16.217.126:36413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:15:57.624614 8347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:15:57.626037 8347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:15:57.643682 8363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:57.643971 8361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:15:57.644124 8359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:15:57.644690 8347 server_base.cc:1034] running on GCE node
I20260812 03:15:57.645256 8347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:15:57.646142 8347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:15:57.647390 8347 hybrid_clock.cc:648] HybridClock initialized: now 1786504557647367 us; error 37 us; skew 500 ppm
Aug 12 03:15:57 dist-test-slave-7nm7 krb5kdc[5963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504557, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:15:57.651242 8347 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:15:57.652455 8347 webserver.cc:460] Webserver started at http://127.16.217.67:38547/ using document root <none> and password file <none>
I20260812 03:15:57.653223 8347 fs_manager.cc:359] Metadata directory not provided
I20260812 03:15:57.653339 8347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:15:57 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504557, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
I20260812 03:15:57.674852 8347 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:15:57.677798 8379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:57.679278 8347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:15:57.679433 8347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "f18883616ef14ec2bfc0d3636e5103ae"
format_stamp: "Formatted at 2026-08-12 03:15:56 on dist-test-slave-7nm7"
server_key: "b3c345eee4bb297e6877aa81c796fa7c"
server_key_iv: "12d5e165fa8483afd10a5eadec3a9f30"
server_key_version: "kuduclusterkey@0"
I20260812 03:15:57.680028 8347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:15:57.719004 8347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:15:57.723065 8347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:15:57.723286 8347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:15:57.724051 8347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:15:57.725133 8347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:15:57.725199 8347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:57.725257 8347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:15:57.725289 8347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:15:57.737378 8347 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:39603
I20260812 03:15:57.737450 8566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:39603 every 8 connection(s)
I20260812 03:15:57.738570 8347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:15:57.742563 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 8347
Aug 12 03:15:57 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504557, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:15:57.757836 8138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:57.740779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:36143 (local address 127.16.217.126:36413)
0812 03:15:57.741287 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:57.741293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:57.742111 (+ 818us) server_negotiation.cc:407] Connection header received
0812 03:15:57.744313 (+ 2202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:57.744316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:57.744366 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:57.744443 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:15:57.746504 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:57.747293 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:57.749446 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:57.749793 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:57.749883 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:57.753076 (+ 3193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:15:57.753080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:15:57.753083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:15:57.753112 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:15:57.754916 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:57.755807 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:57.755811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:57.755812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:57.755905 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:57.756376 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:57.756380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:57.756382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:57.756625 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:15:57.756734 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:57.757501 (+ 767us) server_negotiation.cc:299] Negotiation successful
0812 03:15:57.757643 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355}
I20260812 03:15:57.761617 8571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:57.741374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:36413 (local address 127.16.217.67:36143)
0812 03:15:57.741931 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:57.741993 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:57.744046 (+ 2053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:57.744685 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:57.744696 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:57.745087 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:15:57.746144 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:57.746194 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:57.747796 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:57.747801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:57.749282 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:57.749297 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:57.750111 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:57.750116 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:57.750368 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:57.751114 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:15:57.751135 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:15:57.752856 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:15:57.755099 (+ 2243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:15:57.755106 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:15:57.755126 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:15:57.755652 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:15:57.756024 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:15:57.756027 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:15:57.756029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:15:57.756244 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:15:57.756815 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:15:57.756823 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:15:57.757183 (+ 360us) client_negotiation.cc:769] Sending connection context
0812 03:15:57.759962 (+ 2779us) client_negotiation.cc:240] Negotiation successful
0812 03:15:57.760388 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":132,"threads_started":1}
I20260812 03:15:57.763499 8567 heartbeater.cc:347] Connected to a master server at 127.16.217.126:36413
I20260812 03:15:57.763779 8567 heartbeater.cc:464] Registering TS with master...
I20260812 03:15:57.764395 8567 heartbeater.cc:511] Master 127.16.217.126:36413 requested a full tablet report, sending...
I20260812 03:15:57.765964 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:57.743749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43724 (local address 127.16.217.126:36413)
0812 03:15:57.744010 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:57.744018 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:57.746697 (+ 2679us) server_negotiation.cc:407] Connection header received
0812 03:15:57.746822 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:57.746825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:57.746882 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:57.746994 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:15:57.748960 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:57.749815 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:57.751002 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:57.751374 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:57.751491 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:57.759610 (+ 8119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:15:57.759615 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:15:57.759617 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:15:57.759644 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:15:57.763734 (+ 4090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:57.764341 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:57.764345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:57.764347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:57.764440 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:57.764792 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:57.764796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:57.764798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:57.765042 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:15:57.765244 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:57.765625 (+ 381us) server_negotiation.cc:299] Negotiation successful
0812 03:15:57.765780 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":122,"threads_started":1}
I20260812 03:15:57.766075 7801 ts_manager.cc:194] Registered new tserver with Master: f18883616ef14ec2bfc0d3636e5103ae (127.16.217.67:39603)
I20260812 03:15:57.767503 7801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:36143
Aug 12 03:15:58 dist-test-slave-7nm7 krb5kdc[5963](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786504558, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:15:58.170710 8133 heartbeater.cc:464] Registering TS with master...
I20260812 03:15:58.171090 8133 heartbeater.cc:511] Master 127.16.217.126:36413 requested a full tablet report, sending...
I20260812 03:15:58.172000 7801 ts_manager.cc:194] Registered new tserver with Master: d47b6b336aca4e59b5b41b4b273a6623 (127.16.217.65:42301)
I20260812 03:15:58.172627 7801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:35345
I20260812 03:15:58.489560 8343 heartbeater.cc:464] Registering TS with master...
I20260812 03:15:58.489912 8343 heartbeater.cc:511] Master 127.16.217.126:36413 requested a full tablet report, sending...
I20260812 03:15:58.490840 7801 ts_manager.cc:194] Registered new tserver with Master: da7c473f215c476589a48af5aaa78ba3 (127.16.217.66:41933)
I20260812 03:15:58.491662 7801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:44917
I20260812 03:15:58.500451 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:15:58.515666 8600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:58.502622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43734 (local address 127.16.217.126:36413)
0812 03:15:58.502886 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:58.502891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:58.503182 (+ 291us) server_negotiation.cc:407] Connection header received
0812 03:15:58.503304 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:58.503307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:58.503371 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:58.503465 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:15:58.504884 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.505751 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.507005 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.507384 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.507513 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:58.510893 (+ 3380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:15:58.510897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:15:58.510900 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:15:58.510935 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:15:58.512995 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:58.513570 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:58.513574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:58.513576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:58.513685 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:15:58.514092 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:15:58.514096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:15:58.514097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:15:58.514417 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:15:58.514566 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:58.515320 (+ 754us) server_negotiation.cc:299] Negotiation successful
0812 03:15:58.515468 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":92,"threads_started":1}
I20260812 03:15:58.521661 7801 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43734:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:15:58.527011 7801 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:15:58.553505 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:58.544761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:42301 (local address 127.0.0.1:56712)
0812 03:15:58.546223 (+ 1462us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:58.546317 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:58.546453 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:58.548317 (+ 1864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:58.548323 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:58.548350 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:15:58.548500 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:58.548506 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.550829 (+ 2323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.550833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:58.551938 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:58.551947 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.552999 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.553003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:58.553084 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:58.553120 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 03:15:58.553166 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 03:15:58.553292 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1244,"mutex_wait_us":20,"thread_start_us":129,"threads_started":1}
I20260812 03:15:58.554651 8608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:58.544875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56712 (local address 127.16.217.65:42301)
0812 03:15:58.547596 (+ 2721us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:58.547606 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:58.547634 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:15:58.547721 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:58.547728 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:58.547949 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:58.548104 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:15:58.548640 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.550698 (+ 2058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.552086 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.552869 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.553283 (+ 414us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:58.553422 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:58.553480 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 03:15:58.553594 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2603,"thread_start_us":108,"threads_started":1}
I20260812 03:15:58.556304 8612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:58.545747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:41933 (local address 127.0.0.1:38766)
0812 03:15:58.547473 (+ 1726us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:58.547486 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:58.547571 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:58.548462 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:58.548465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:58.548479 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:15:58.548574 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:58.548580 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.550023 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.550026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:58.551507 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:58.551519 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.552837 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.552841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:58.552978 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:58.553026 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 03:15:58.553134 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 03:15:58.556150 (+ 3016us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1653,"mutex_wait_us":4,"thread_start_us":113,"threads_started":1}
I20260812 03:15:58.558158 8015 tablet_service.cc:1467] Processing CreateTablet for tablet 7ad58a99a1e74058915c156c418112a9 (DEFAULT_TABLE table=test-table [id=dca10b1fb40448038a1351568e5c1cbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:15:58.559594 8015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ad58a99a1e74058915c156c418112a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:15:58.559991 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:58.546161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:39603 (local address 127.0.0.1:58172)
0812 03:15:58.554205 (+ 8044us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:58.554221 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:58.554325 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:58.555249 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:58.555253 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:58.555271 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:15:58.555345 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:58.555350 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.556919 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.556922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:58.558119 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:58.558135 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.559538 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.559543 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:58.559637 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:58.559665 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 03:15:58.559766 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 03:15:58.559854 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7989,"thread_start_us":106,"threads_started":1}
I20260812 03:15:58.560577 8616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:58.547071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58172 (local address 127.16.217.67:39603)
0812 03:15:58.548408 (+ 1337us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:58.548414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:58.554438 (+ 6024us) server_negotiation.cc:407] Connection header received
0812 03:15:58.554533 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:58.554540 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:58.554925 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:58.555089 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:15:58.555474 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.556779 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.558314 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.559201 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.559319 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:58.559468 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:58.560317 (+ 849us) server_negotiation.cc:299] Negotiation successful
0812 03:15:58.560429 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1240,"thread_start_us":128,"threads_started":1}
I20260812 03:15:58.562091 8613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:58.546744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38766 (local address 127.16.217.66:41933)
0812 03:15:58.547795 (+ 1051us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:58.547801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:58.547820 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:15:58.547884 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:58.547888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:58.548043 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:58.548161 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:15:58.548696 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.549881 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.551650 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.552483 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.552591 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:58.552762 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:58.561802 (+ 9040us) server_negotiation.cc:299] Negotiation successful
0812 03:15:58.561946 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":959,"thread_start_us":77,"threads_started":1}
I20260812 03:15:58.565232 8232 tablet_service.cc:1467] Processing CreateTablet for tablet 7ad58a99a1e74058915c156c418112a9 (DEFAULT_TABLE table=test-table [id=dca10b1fb40448038a1351568e5c1cbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:15:58.566462 8232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ad58a99a1e74058915c156c418112a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:15:58.569882 8619 tablet_bootstrap.cc:492] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623: Bootstrap starting.
I20260812 03:15:58.570200 8453 tablet_service.cc:1467] Processing CreateTablet for tablet 7ad58a99a1e74058915c156c418112a9 (DEFAULT_TABLE table=test-table [id=dca10b1fb40448038a1351568e5c1cbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:15:58.571463 8453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ad58a99a1e74058915c156c418112a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:15:58.578775 8619 tablet_bootstrap.cc:654] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623: Neither blocks nor log segments found. Creating new log.
I20260812 03:15:58.579795 8619 log.cc:824] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623: Log is configured to *not* fsync() on all Append() calls
I20260812 03:15:58.580696 8620 tablet_bootstrap.cc:492] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3: Bootstrap starting.
I20260812 03:15:58.583765 8620 tablet_bootstrap.cc:654] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3: Neither blocks nor log segments found. Creating new log.
I20260812 03:15:58.583837 8619 tablet_bootstrap.cc:492] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623: No bootstrap required, opened a new log
I20260812 03:15:58.584116 8619 ts_tablet_manager.cc:1397] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.000s
I20260812 03:15:58.584780 8620 log.cc:824] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:15:58.585623 8621 tablet_bootstrap.cc:492] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae: Bootstrap starting.
I20260812 03:15:58.587944 8621 tablet_bootstrap.cc:654] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae: Neither blocks nor log segments found. Creating new log.
I20260812 03:15:58.588515 8619 raft_consensus.cc:348] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d47b6b336aca4e59b5b41b4b273a6623" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42301 } } peers { permanent_uuid: "da7c473f215c476589a48af5aaa78ba3" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41933 } } peers { permanent_uuid: "f18883616ef14ec2bfc0d3636e5103ae" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39603 } }
I20260812 03:15:58.588835 8621 log.cc:824] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae: Log is configured to *not* fsync() on all Append() calls
I20260812 03:15:58.588941 8619 raft_consensus.cc:374] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:15:58.589126 8619 raft_consensus.cc:729] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d47b6b336aca4e59b5b41b4b273a6623, State: Initialized, Role: FOLLOWER
I20260812 03:15:58.589784 8619 consensus_queue.cc:260] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d47b6b336aca4e59b5b41b4b273a6623" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42301 } } peers { permanent_uuid: "da7c473f215c476589a48af5aaa78ba3" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41933 } } peers { permanent_uuid: "f18883616ef14ec2bfc0d3636e5103ae" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39603 } }
I20260812 03:15:58.591091 8619 ts_tablet_manager.cc:1428] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260812 03:15:58.591531 8133 heartbeater.cc:503] Master 127.16.217.126:36413 was elected leader, sending a full tablet report...
I20260812 03:15:58.591768 8620 tablet_bootstrap.cc:492] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3: No bootstrap required, opened a new log
I20260812 03:15:58.591966 8620 ts_tablet_manager.cc:1397] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260812 03:15:58.595906 8620 raft_consensus.cc:348] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d47b6b336aca4e59b5b41b4b273a6623" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42301 } } peers { permanent_uuid: "da7c473f215c476589a48af5aaa78ba3" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41933 } } peers { permanent_uuid: "f18883616ef14ec2bfc0d3636e5103ae" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39603 } }
I20260812 03:15:58.596185 8620 raft_consensus.cc:374] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:15:58.596288 8620 raft_consensus.cc:729] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7c473f215c476589a48af5aaa78ba3, State: Initialized, Role: FOLLOWER
I20260812 03:15:58.596768 8620 consensus_queue.cc:260] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d47b6b336aca4e59b5b41b4b273a6623" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42301 } } peers { permanent_uuid: "da7c473f215c476589a48af5aaa78ba3" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41933 } } peers { permanent_uuid: "f18883616ef14ec2bfc0d3636e5103ae" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39603 } }
I20260812 03:15:58.597790 8620 ts_tablet_manager.cc:1428] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 03:15:58.598183 8343 heartbeater.cc:503] Master 127.16.217.126:36413 was elected leader, sending a full tablet report...
I20260812 03:15:58.601294 8621 tablet_bootstrap.cc:492] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae: No bootstrap required, opened a new log
I20260812 03:15:58.601505 8621 ts_tablet_manager.cc:1397] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.000s
I20260812 03:15:58.605783 8621 raft_consensus.cc:348] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d47b6b336aca4e59b5b41b4b273a6623" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42301 } } peers { permanent_uuid: "da7c473f215c476589a48af5aaa78ba3" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41933 } } peers { permanent_uuid: "f18883616ef14ec2bfc0d3636e5103ae" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39603 } }
I20260812 03:15:58.606058 8621 raft_consensus.cc:374] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:15:58.606102 8621 raft_consensus.cc:729] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f18883616ef14ec2bfc0d3636e5103ae, State: Initialized, Role: FOLLOWER
I20260812 03:15:58.606608 8621 consensus_queue.cc:260] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d47b6b336aca4e59b5b41b4b273a6623" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42301 } } peers { permanent_uuid: "da7c473f215c476589a48af5aaa78ba3" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41933 } } peers { permanent_uuid: "f18883616ef14ec2bfc0d3636e5103ae" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39603 } }
I20260812 03:15:58.607671 8621 ts_tablet_manager.cc:1428] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:15:58.608069 8567 heartbeater.cc:503] Master 127.16.217.126:36413 was elected leader, sending a full tablet report...
Aug 12 03:15:58 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504558, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
W20260812 03:15:58.631101 8134 tablet.cc:2369] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:15:58.725816 8344 tablet.cc:2369] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:15:58.740140 8570 tablet.cc:2369] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 03:15:58 dist-test-slave-7nm7 krb5kdc[5963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504558, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for HTTP/127.16.217.84@KRBTEST.COM
I20260812 03:15:58.903728 8628 raft_consensus.cc:482] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:15:58.903975 8628 raft_consensus.cc:504] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d47b6b336aca4e59b5b41b4b273a6623" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42301 } } peers { permanent_uuid: "da7c473f215c476589a48af5aaa78ba3" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41933 } } peers { permanent_uuid: "f18883616ef14ec2bfc0d3636e5103ae" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39603 } }
I20260812 03:15:58.905392 8628 leader_election.cc:288] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da7c473f215c476589a48af5aaa78ba3 (127.16.217.66:41933), f18883616ef14ec2bfc0d3636e5103ae (127.16.217.67:39603)
I20260812 03:15:58.912102 8613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:58.906553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:41755 (local address 127.16.217.66:41933)
0812 03:15:58.906878 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:58.906886 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:58.907441 (+ 555us) server_negotiation.cc:407] Connection header received
0812 03:15:58.907662 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:58.907666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:58.907717 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:58.907767 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:15:58.908248 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.909037 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.910799 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.911412 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.911509 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:58.911563 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:58.911896 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 03:15:58.911973 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224}
I20260812 03:15:58.912833 8634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:58.905907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:41933 (local address 127.16.217.65:41755)
0812 03:15:58.907346 (+ 1439us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:58.907363 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:58.907533 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:58.907919 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:58.907922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:58.907942 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:15:58.908090 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:58.908097 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.909241 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.909246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:58.910651 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:58.910663 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.911660 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.911663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:58.911762 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:58.911785 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:15:58.912605 (+ 820us) client_negotiation.cc:240] Negotiation successful
0812 03:15:58.912685 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1361,"thread_start_us":137,"threads_started":1}
I20260812 03:15:58.913717 8266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad58a99a1e74058915c156c418112a9" candidate_uuid: "d47b6b336aca4e59b5b41b4b273a6623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7c473f215c476589a48af5aaa78ba3" is_pre_election: true
I20260812 03:15:58.914054 8266 raft_consensus.cc:2454] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d47b6b336aca4e59b5b41b4b273a6623 in term 0.
I20260812 03:15:58.914695 7945 leader_election.cc:302] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d47b6b336aca4e59b5b41b4b273a6623, da7c473f215c476589a48af5aaa78ba3; no voters:
I20260812 03:15:58.915014 8628 raft_consensus.cc:2781] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:15:58.915083 8628 raft_consensus.cc:482] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:15:58.915118 8628 raft_consensus.cc:3037] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:15:58.916543 8628 raft_consensus.cc:504] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d47b6b336aca4e59b5b41b4b273a6623" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42301 } } peers { permanent_uuid: "da7c473f215c476589a48af5aaa78ba3" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41933 } } peers { permanent_uuid: "f18883616ef14ec2bfc0d3636e5103ae" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39603 } }
I20260812 03:15:58.916913 8628 leader_election.cc:288] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [CANDIDATE]: Term 1 election: Requested vote from peers da7c473f215c476589a48af5aaa78ba3 (127.16.217.66:41933), f18883616ef14ec2bfc0d3636e5103ae (127.16.217.67:39603)
I20260812 03:15:58.917536 8266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad58a99a1e74058915c156c418112a9" candidate_uuid: "d47b6b336aca4e59b5b41b4b273a6623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7c473f215c476589a48af5aaa78ba3"
I20260812 03:15:58.917657 8266 raft_consensus.cc:3037] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:15:58.919051 8266 raft_consensus.cc:2454] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d47b6b336aca4e59b5b41b4b273a6623 in term 1.
I20260812 03:15:58.919796 7945 leader_election.cc:302] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d47b6b336aca4e59b5b41b4b273a6623, da7c473f215c476589a48af5aaa78ba3; no voters:
I20260812 03:15:58.920138 8628 raft_consensus.cc:2781] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:15:58.920406 8628 raft_consensus.cc:686] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [term 1 LEADER]: Becoming Leader. State: Replica: d47b6b336aca4e59b5b41b4b273a6623, State: Running, Role: LEADER
I20260812 03:15:58.920809 8628 consensus_queue.cc:237] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d47b6b336aca4e59b5b41b4b273a6623" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42301 } } peers { permanent_uuid: "da7c473f215c476589a48af5aaa78ba3" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41933 } } peers { permanent_uuid: "f18883616ef14ec2bfc0d3636e5103ae" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39603 } }
I20260812 03:15:58.922875 8616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:58.906715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:37935 (local address 127.16.217.67:39603)
0812 03:15:58.907018 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:58.907024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:58.914785 (+ 7761us) server_negotiation.cc:407] Connection header received
0812 03:15:58.915024 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:58.915028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:58.915092 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:58.915153 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:15:58.915630 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.916341 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.921232 (+ 4891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.921923 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.922035 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:58.922093 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:58.922613 (+ 520us) server_negotiation.cc:299] Negotiation successful
0812 03:15:58.922725 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189}
I20260812 03:15:58.923518 8638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:58.906487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:39603 (local address 127.16.217.65:37935)
0812 03:15:58.914672 (+ 8185us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:58.914691 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:15:58.914875 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:15:58.915312 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:15:58.915315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:15:58.915334 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:15:58.915478 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:58.915485 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.916529 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.916533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:58.917685 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:15:58.917698 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:15:58.922326 (+ 4628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:15:58.922330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:15:58.922449 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:58.922474 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:15:58.923269 (+ 795us) client_negotiation.cc:240] Negotiation successful
0812 03:15:58.923362 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8105,"thread_start_us":115,"threads_started":1}
I20260812 03:15:58.924378 8486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad58a99a1e74058915c156c418112a9" candidate_uuid: "d47b6b336aca4e59b5b41b4b273a6623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f18883616ef14ec2bfc0d3636e5103ae" is_pre_election: true
I20260812 03:15:58.924705 8486 raft_consensus.cc:2454] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d47b6b336aca4e59b5b41b4b273a6623 in term 0.
I20260812 03:15:58.924387 8491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ad58a99a1e74058915c156c418112a9" candidate_uuid: "d47b6b336aca4e59b5b41b4b273a6623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f18883616ef14ec2bfc0d3636e5103ae"
I20260812 03:15:58.925531 8491 raft_consensus.cc:3037] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:15:58.925590 7801 catalog_manager.cc:5495] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 reported cstate change: term changed from 0 to 1, leader changed from <none> to d47b6b336aca4e59b5b41b4b273a6623 (127.16.217.65). New cstate: current_term: 1 leader_uuid: "d47b6b336aca4e59b5b41b4b273a6623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d47b6b336aca4e59b5b41b4b273a6623" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da7c473f215c476589a48af5aaa78ba3" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f18883616ef14ec2bfc0d3636e5103ae" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39603 } health_report { overall_health: UNKNOWN } } }
I20260812 03:15:58.926932 8491 raft_consensus.cc:2454] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d47b6b336aca4e59b5b41b4b273a6623 in term 1.
I20260812 03:15:59.016544 8608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:59.011359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56720 (local address 127.16.217.65:42301)
0812 03:15:59.011517 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 03:15:59.011522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:15:59.011538 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:15:59.011647 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:15:59.011650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:15:59.011744 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:15:59.011878 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:15:59.012228 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:59.012998 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:59.014218 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:15:59.014577 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:15:59.014671 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:15:59.015120 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:15:59.015729 (+ 609us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:15:59.015851 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 03:15:59.016292 (+ 441us) server_negotiation.cc:299] Negotiation successful
0812 03:15:59.016368 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:15:59.025437 8491 raft_consensus.cc:1261] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae [term 1 FOLLOWER]: Refusing update from remote peer d47b6b336aca4e59b5b41b4b273a6623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:15:59.026445 8639 consensus_queue.cc:1035] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f18883616ef14ec2bfc0d3636e5103ae" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:15:59.034080 8266 raft_consensus.cc:1261] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3 [term 1 FOLLOWER]: Refusing update from remote peer d47b6b336aca4e59b5b41b4b273a6623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:15:59.034973 8639 consensus_queue.cc:1035] T 7ad58a99a1e74058915c156c418112a9 P d47b6b336aca4e59b5b41b4b273a6623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da7c473f215c476589a48af5aaa78ba3" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 41933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:15:59.060524 8653 mvcc.cc:205] Tried to move back new op lower bound from 7317522673757720576 to 7317522673347743744. Current Snapshot: MvccSnapshot[applied={T|T < 7317522673757720576}]
I20260812 03:15:59.070914 7801 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:15:59.071139 7801 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:15:59.074120 7801 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 410521a341c04193b3640564c2439a50: Sending DeleteTablet for 3 replicas of tablet 7ad58a99a1e74058915c156c418112a9
I20260812 03:15:59.075297 8015 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ad58a99a1e74058915c156c418112a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:15:59 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:56712
I20260812 03:15:59.075575 8232 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ad58a99a1e74058915c156c418112a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:15:59 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:38766
I20260812 03:15:59.075728 8453 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ad58a99a1e74058915c156c418112a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:15:59 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:58172
I20260812 03:15:59.076611 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 7858
I20260812 03:15:59.076624 8669 tablet_replica.cc:330] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae: stopping tablet replica
I20260812 03:15:59.076946 8669 raft_consensus.cc:2229] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:15:59.077195 8669 raft_consensus.cc:2258] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:15:59.081151 8668 tablet_replica.cc:330] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3: stopping tablet replica
I20260812 03:15:59.081804 8668 raft_consensus.cc:2229] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:15:59.082203 8668 raft_consensus.cc:2258] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:15:59.098995 8668 ts_tablet_manager.cc:1905] T 7ad58a99a1e74058915c156c418112a9 P da7c473f215c476589a48af5aaa78ba3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:15:59.102470 8669 ts_tablet_manager.cc:1905] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 03:15:59.104532 7763 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.217.65:42301 (error 108)
I20260812 03:15:59.105571 8669 ts_tablet_manager.cc:1918] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:15:59.105688 8669 log.cc:1196] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/7ad58a99a1e74058915c156c418112a9
I20260812 03:15:59.105670 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 8140
I20260812 03:15:59.106019 8669 ts_tablet_manager.cc:1939] T 7ad58a99a1e74058915c156c418112a9 P f18883616ef14ec2bfc0d3636e5103ae: Deleting consensus metadata
I20260812 03:15:59.112907 7765 catalog_manager.cc:4841] TS f18883616ef14ec2bfc0d3636e5103ae (127.16.217.67:39603): tablet 7ad58a99a1e74058915c156c418112a9 (table test-table [id=dca10b1fb40448038a1351568e5c1cbd]) successfully deleted
I20260812 03:15:59.137825 8671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:59.137280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:42301 (local address 127.0.0.1:56730)
0812 03:15:59.137556 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:59.137662 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:42301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":106,"threads_started":1}
W20260812 03:15:59.138206 7763 catalog_manager.cc:4584] TS d47b6b336aca4e59b5b41b4b273a6623 (127.16.217.65:42301): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7ad58a99a1e74058915c156c418112a9: Network error: Client connection negotiation failed: client connection to 127.16.217.65:42301: connect: Connection refused (error 111)
I20260812 03:15:59.146970 8671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:59.146597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:41933 (local address 127.0.0.1:38782)
0812 03:15:59.146811 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:59.146868 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.66:41933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260812 03:15:59.147732 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 8347
I20260812 03:15:59.220899 8671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:59.213518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:42301 (local address 127.0.0.1:56738)
0812 03:15:59.220654 (+ 7136us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:59.220768 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:42301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6999}
I20260812 03:15:59.221266 8671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:59.216016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:41933 (local address 127.0.0.1:38784)
0812 03:15:59.221129 (+ 5113us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:59.221185 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.66:41933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5041,"thread_start_us":83,"threads_started":1}
I20260812 03:15:59.222545 8674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:59.221838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:42301 (local address 127.0.0.1:56750)
0812 03:15:59.222408 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:59.222467 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:42301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496}
I20260812 03:15:59.226317 8674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:15:59.222731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:41933 (local address 127.0.0.1:38792)
0812 03:15:59.226151 (+ 3420us) negotiation.cc:107] Waiting for socket to connect
0812 03:15:59.226220 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.66:41933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3345}
I20260812 03:15:59.227252 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 7730
I20260812 03:15:59.247406 17253 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 03:15:59.593343 17253 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 03:15:59.593681 17253 mini_ranger.cc:66] Stopping Ranger...
I20260812 03:15:59.950039 17253 mini_ranger.cc:68] Stopped Ranger
2026-08-12 03:15:59.950 UTC [6013] LOG: received smart shutdown request
2026-08-12 03:15:59.955 UTC [6013] LOG: background worker "logical replication launcher" (PID 6036) exited with exit code 1
2026-08-12 03:15:59.955 UTC [6026] LOG: shutting down
2026-08-12 03:16:00.150 UTC [6013] LOG: database system is shut down
2026-08-12T03:16:00Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53672 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:16:00 dist-test-slave-7nm7 krb5kdc[8695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:16:00 dist-test-slave-7nm7 krb5kdc[8695](info): set up 2 sockets
Aug 12 03:16:00 dist-test-slave-7nm7 krb5kdc[8695](info): commencing operation
krb5kdc: starting...
W20260812 03:16:01.619817 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:16:01 dist-test-slave-7nm7 krb5kdc[8695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:16:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:16:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:16:01.758553 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:39911
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:38227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:39911,127.16.217.125:38267,127.16.217.124:35449
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:16:01.892282 8719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:01.892774 8719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:01.892870 8719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:01.897238 8719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:16:01.897571 8719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:01.897811 8719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:01.897881 8719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:16:01.897924 8719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:16:01.903957 8719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:38227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:39911,127.16.217.125:38267,127.16.217.124:35449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:39911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:01.905252 8719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:01.906373 8719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:01.916123 8734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:01.916146 8731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:01.916124 8730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:01.916185 8719 server_base.cc:1034] running on GCE node
I20260812 03:16:01.917083 8719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:01.918817 8719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:01.920068 8719 hybrid_clock.cc:648] HybridClock initialized: now 1786504561919984 us; error 105 us; skew 500 ppm
Aug 12 03:16:01 dist-test-slave-7nm7 krb5kdc[8695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504561, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:01.923987 8719 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:16:01.925329 8719 webserver.cc:460] Webserver started at http://127.16.217.126:41121/ using document root <none> and password file <none>
I20260812 03:16:01.925943 8719 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:01.926043 8719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:01.926360 8719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:16:01.928516 8719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "649df843964f44cf80ffc1e17abaf7c3"
format_stamp: "Formatted at 2026-08-12 03:16:01 on dist-test-slave-7nm7"
server_key: "21b3846eef79aea0adf5a9a130521844"
server_key_iv: "238b4ab2e243153b55d97ce7865523b5"
server_key_version: "encryptionkey@0"
I20260812 03:16:01.929116 8719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "649df843964f44cf80ffc1e17abaf7c3"
format_stamp: "Formatted at 2026-08-12 03:16:01 on dist-test-slave-7nm7"
server_key: "21b3846eef79aea0adf5a9a130521844"
server_key_iv: "238b4ab2e243153b55d97ce7865523b5"
server_key_version: "encryptionkey@0"
I20260812 03:16:01.932909 8719 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 03:16:01.935897 8744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:01.937386 8719 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:16:01.937577 8719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "649df843964f44cf80ffc1e17abaf7c3"
format_stamp: "Formatted at 2026-08-12 03:16:01 on dist-test-slave-7nm7"
server_key: "21b3846eef79aea0adf5a9a130521844"
server_key_iv: "238b4ab2e243153b55d97ce7865523b5"
server_key_version: "encryptionkey@0"
I20260812 03:16:01.937706 8719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:01.962414 8719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:01.966403 8719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:01.966635 8719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:01.974472 8719 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:39911
I20260812 03:16:01.974826 8823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:39911 every 8 connection(s)
I20260812 03:16:01.975534 8719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:16:01.978593 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 8719
I20260812 03:16:01.978739 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:16:01.978765 8825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:16:01.979044 17253 external_mini_cluster.cc:1428] Setting key 0b99ae44c553848a87df838b1a78326e
I20260812 03:16:01.984088 8825 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.16.217.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.125@KRBTEST.COM" created.
Aug 12 03:16:01 dist-test-slave-7nm7 krb5kdc[8695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504561, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:16:02.002506 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:01.986570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39911 (local address 127.0.0.1:36892)
0812 03:16:01.987205 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:01.987264 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:01.988201 (+ 937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:01.989091 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:01.989097 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:01.989530 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:01.989802 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:01.989815 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:01.991095 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:01.991097 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:01.992110 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:01.992128 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:01.992926 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:01.992933 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:01.993126 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:01.993868 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:01.993895 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:01.995895 (+ 2000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:01.998511 (+ 2616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:01.998521 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:01.998530 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:01.998818 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:01.999258 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:01.999261 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:01.999262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:01.999404 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:02.000492 (+ 1088us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:02.000502 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:02.000779 (+ 277us) client_negotiation.cc:769] Sending connection context
0812 03:16:02.001037 (+ 258us) client_negotiation.cc:240] Negotiation successful
0812 03:16:02.001584 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":64,"thread_start_us":159,"threads_started":1}
I20260812 03:16:02.002509 8831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:01.986572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36892 (local address 127.16.217.126:39911)
0812 03:16:01.987265 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:01.987284 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:01.987393 (+ 109us) server_negotiation.cc:407] Connection header received
0812 03:16:01.988549 (+ 1156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:01.988559 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:01.988843 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:01.989058 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:01.989952 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:01.990937 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:01.992239 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:01.992742 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:01.992874 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:01.996064 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:01.996075 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:01.996092 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:01.996124 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:01.998367 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:01.998957 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:01.998966 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:01.998970 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:01.999130 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:01.999542 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:01.999548 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:01.999552 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.000198 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:02.000365 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:02.001188 (+ 823us) server_negotiation.cc:299] Negotiation successful
0812 03:16:02.001477 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"spinlock_wait_cycles":15104,"thread_start_us":159,"threads_started":1}
I20260812 03:16:02.005108 8825 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:16:02.005946 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.005665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38267 (local address 127.0.0.1:48306)
0812 03:16:02.005801 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.005881 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:38267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 03:16:02.006199 8749 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.217.125:38267: connect: Connection refused (error 111)
I20260812 03:16:02.007339 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.007050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38267 (local address 127.0.0.1:48318)
0812 03:16:02.007194 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.007245 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:38267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260812 03:16:02.007582 8825 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.125:38267: Network error: Client connection negotiation failed: client connection to 127.16.217.125:38267: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.125.keytab.
Entry for principal kudu/127.16.217.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.125.keytab.
I20260812 03:16:02.012110 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.125:38267
--webserver_interface=127.16.217.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.125.keytab
--principal=kudu/127.16.217.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:38227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:39911,127.16.217.125:38267,127.16.217.124:35449
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:16:02.056874 8825 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } attempt: 1
I20260812 03:16:02.058177 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.057532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38267 (local address 127.0.0.1:48322)
0812 03:16:02.057935 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.058033 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:38267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260812 03:16:02.059137 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.058852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38267 (local address 127.0.0.1:48326)
0812 03:16:02.059006 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.059071 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:38267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 03:16:02.059439 8825 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.125:38267: Network error: Client connection negotiation failed: client connection to 127.16.217.125:38267: connect: Connection refused (error 111)
I20260812 03:16:02.127785 8825 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } attempt: 2
I20260812 03:16:02.129216 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.128705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38267 (local address 127.0.0.1:48336)
0812 03:16:02.129020 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.129114 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:38267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
I20260812 03:16:02.130103 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.129865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38267 (local address 127.0.0.1:48352)
0812 03:16:02.129983 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.130040 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:38267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 03:16:02.130340 8825 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.125:38267: Network error: Client connection negotiation failed: client connection to 127.16.217.125:38267: connect: Connection refused (error 111)
W20260812 03:16:02.144335 8834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:02.144678 8834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:02.144807 8834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:02.149236 8834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:16:02.149363 8834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:02.149405 8834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:02.149446 8834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:16:02.149479 8834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:16:02.155138 8834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:38227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:39911,127.16.217.125:38267,127.16.217.124:35449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.125.keytab
--principal=kudu/127.16.217.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.125:38267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.217.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:02.156383 8834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:02.157727 8834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:02.169076 8841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:02.169153 8847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:02.169153 8840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:02.169585 8834 server_base.cc:1034] running on GCE node
I20260812 03:16:02.170522 8834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:02.171770 8834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:02.172972 8834 hybrid_clock.cc:648] HybridClock initialized: now 1786504562172955 us; error 31 us; skew 500 ppm
Aug 12 03:16:02 dist-test-slave-7nm7 krb5kdc[8695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504562, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:02.176683 8834 init.cc:377] Logged in from keytab as kudu/127.16.217.125@KRBTEST.COM (short username kudu)
I20260812 03:16:02.178203 8834 webserver.cc:460] Webserver started at http://127.16.217.125:38707/ using document root <none> and password file <none>
I20260812 03:16:02.179054 8834 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:02.179138 8834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:02.179386 8834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:16:02.182098 8834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-1/data/instance:
uuid: "4af0c8a29291422bb2614eceb32fdda4"
format_stamp: "Formatted at 2026-08-12 03:16:02 on dist-test-slave-7nm7"
server_key: "66ca47777038ea46071926682ea417aa"
server_key_iv: "ba8dce35bfc59bc8de49155a8f4912d1"
server_key_version: "encryptionkey@0"
I20260812 03:16:02.182894 8834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-1/wal/instance:
uuid: "4af0c8a29291422bb2614eceb32fdda4"
format_stamp: "Formatted at 2026-08-12 03:16:02 on dist-test-slave-7nm7"
server_key: "66ca47777038ea46071926682ea417aa"
server_key_iv: "ba8dce35bfc59bc8de49155a8f4912d1"
server_key_version: "encryptionkey@0"
I20260812 03:16:02.188015 8834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 03:16:02.191033 8859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:02.193010 8834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:16:02.193203 8834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-1/wal
uuid: "4af0c8a29291422bb2614eceb32fdda4"
format_stamp: "Formatted at 2026-08-12 03:16:02 on dist-test-slave-7nm7"
server_key: "66ca47777038ea46071926682ea417aa"
server_key_iv: "ba8dce35bfc59bc8de49155a8f4912d1"
server_key_version: "encryptionkey@0"
I20260812 03:16:02.193341 8834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:02.211669 8834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:02.215623 8834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:02.215853 8834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:02.221522 8825 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } attempt: 3
I20260812 03:16:02.222752 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.222322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38267 (local address 127.0.0.1:48362)
0812 03:16:02.222555 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.222652 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:38267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 03:16:02.223899 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.223594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38267 (local address 127.0.0.1:48366)
0812 03:16:02.223745 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.223816 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:38267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 03:16:02.224153 8825 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.125:38267: Network error: Client connection negotiation failed: client connection to 127.16.217.125:38267: connect: Connection refused (error 111)
I20260812 03:16:02.225390 8834 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.125:38267
I20260812 03:16:02.225704 8952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.125:38267 every 8 connection(s)
I20260812 03:16:02.226723 8834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-1/data/info.pb
I20260812 03:16:02.229861 8953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:16:02.231876 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 8834
I20260812 03:16:02.232033 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-1/wal/instance
I20260812 03:16:02.232693 17253 external_mini_cluster.cc:1428] Setting key 4ce06d5d5a12c06c2d330c42048e3d80
I20260812 03:16:02.235800 8953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.16.217.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.124@KRBTEST.COM" created.
Aug 12 03:16:02 dist-test-slave-7nm7 krb5kdc[8695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504562, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.125@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:16:02.252964 8831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.238087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36900 (local address 127.16.217.126:39911)
0812 03:16:02.238310 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:02.238314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:02.240067 (+ 1753us) server_negotiation.cc:407] Connection header received
0812 03:16:02.241692 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:02.241696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:02.241768 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:02.241875 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:02.243283 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.243885 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.245116 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.245364 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.245479 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.248325 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:02.248329 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:02.248332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:02.248360 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:02.250657 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.251248 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.251257 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.251258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.251336 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.251714 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.251718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.251721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.251928 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:02.252042 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:02.252641 (+ 599us) server_negotiation.cc:299] Negotiation successful
0812 03:16:02.252781 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:16:02.253846 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.238914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39911 (local address 127.0.0.1:36900)
0812 03:16:02.239872 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.239926 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:02.241406 (+ 1480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:02.242085 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:02.242094 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:02.242586 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:02.243124 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:02.243140 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.244036 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.244040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:02.244973 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:02.244983 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.245545 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.245549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:02.245735 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.246426 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:02.246444 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:02.248016 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:02.250799 (+ 2783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:02.250806 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:02.250833 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:02.251127 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:02.251435 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:02.251438 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:02.251440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:02.251604 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:02.252042 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:02.252050 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:02.252365 (+ 315us) client_negotiation.cc:769] Sending connection context
0812 03:16:02.252568 (+ 203us) client_negotiation.cc:240] Negotiation successful
0812 03:16:02.252943 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":209,"threads_started":1}
I20260812 03:16:02.255543 8953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:16:02 dist-test-slave-7nm7 krb5kdc[8695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504562, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.125@KRBTEST.COM for kudu/127.16.217.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.124.keytab.
Entry for principal kudu/127.16.217.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.124.keytab.
I20260812 03:16:02.267959 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.124:35449
--webserver_interface=127.16.217.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.124.keytab
--principal=kudu/127.16.217.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:38227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:39911,127.16.217.125:38267,127.16.217.124:35449
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:16:02.268627 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.256164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38267 (local address 127.0.0.1:48376)
0812 03:16:02.256636 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.256656 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:02.256773 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:02.257348 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:02.257351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:02.257539 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:02.257630 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:02.257635 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.258702 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.258706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:02.259238 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:02.259245 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.259842 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.259846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:02.260055 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.260901 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:02.260916 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:02.262979 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:02.266428 (+ 3449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:02.266432 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:02.266435 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:02.266693 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:02.267311 (+ 618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:02.267314 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:02.267316 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:02.267421 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:02.268028 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:02.268031 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:02.268128 (+ 97us) client_negotiation.cc:769] Sending connection context
0812 03:16:02.268250 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 03:16:02.268412 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":47}
I20260812 03:16:02.268649 8959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.256294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48376 (local address 127.16.217.125:38267)
0812 03:16:02.256620 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:02.256632 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:02.256740 (+ 108us) server_negotiation.cc:407] Connection header received
0812 03:16:02.256880 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:02.256890 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:02.257113 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:02.257374 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:02.257767 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.258583 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.259355 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.259727 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.259845 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.263147 (+ 3302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:02.263158 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:02.263173 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:02.263194 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:02.266138 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.266927 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.266940 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.266945 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.267104 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.267578 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.267582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.267586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.267907 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:02.268076 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:02.268353 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 03:16:02.268498 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":217,"thread_start_us":134,"threads_started":1}
I20260812 03:16:02.271205 8953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:16:02.272557 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.271984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35449 (local address 127.0.0.1:51626)
0812 03:16:02.272197 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.272299 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260812 03:16:02.272876 8864 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.217.124:35449: connect: Connection refused (error 111)
I20260812 03:16:02.274425 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.273761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35449 (local address 127.0.0.1:51636)
0812 03:16:02.274038 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.274140 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
W20260812 03:16:02.274665 8953 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.124:35449: Network error: Client connection negotiation failed: client connection to 127.16.217.124:35449: connect: Connection refused (error 111)
I20260812 03:16:02.323943 8953 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } attempt: 1
I20260812 03:16:02.325191 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.324760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35449 (local address 127.0.0.1:51652)
0812 03:16:02.324990 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.325097 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260812 03:16:02.325994 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.325756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35449 (local address 127.0.0.1:51658)
0812 03:16:02.325887 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.325941 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 03:16:02.326146 8953 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.124:35449: Network error: Client connection negotiation failed: client connection to 127.16.217.124:35449: connect: Connection refused (error 111)
I20260812 03:16:02.367422 8825 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } attempt: 4
Aug 12 03:16:02 dist-test-slave-7nm7 krb5kdc[8695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504561, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for kudu/127.16.217.125@KRBTEST.COM
I20260812 03:16:02.378278 8959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.368225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48380 (local address 127.16.217.125:38267)
0812 03:16:02.368364 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:02.368369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:02.368456 (+ 87us) server_negotiation.cc:407] Connection header received
0812 03:16:02.368647 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:02.368651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:02.368708 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:02.368804 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:02.369292 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.369843 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.370752 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.371598 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.371761 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.374696 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:02.374700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:02.374703 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:02.374745 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:02.376438 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.376965 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.376969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.376971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.377057 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.377348 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.377353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.377355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.377551 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:02.377672 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:02.377939 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 03:16:02.378058 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:16:02.378281 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.368132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38267 (local address 127.0.0.1:48380)
0812 03:16:02.368336 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.368356 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:02.368516 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:02.368824 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:02.368828 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:02.369043 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:02.369161 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:02.369168 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.369977 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.369981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:02.370620 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:02.370630 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.371765 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.371770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:02.371895 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.372552 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:02.372565 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:02.374437 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:02.376609 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:02.376614 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:02.376619 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:02.376850 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:02.377158 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:02.377161 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:02.377163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:02.377249 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:02.377663 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:02.377667 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:02.377771 (+ 104us) client_negotiation.cc:769] Sending connection context
0812 03:16:02.377885 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 03:16:02.378066 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260812 03:16:02.379447 8825 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:16:02.380214 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.379957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35449 (local address 127.0.0.1:51664)
0812 03:16:02.380082 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.380136 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 03:16:02.380992 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.380652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35449 (local address 127.0.0.1:51676)
0812 03:16:02.380866 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.380937 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 03:16:02.381152 8825 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.124:35449: Network error: Client connection negotiation failed: client connection to 127.16.217.124:35449: connect: Connection refused (error 111)
I20260812 03:16:02.394352 8953 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } attempt: 2
I20260812 03:16:02.395169 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.394872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35449 (local address 127.0.0.1:51678)
0812 03:16:02.395021 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.395097 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 03:16:02.395944 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.395747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35449 (local address 127.0.0.1:51684)
0812 03:16:02.395850 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.395893 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 03:16:02.396165 8953 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.124:35449: Network error: Client connection negotiation failed: client connection to 127.16.217.124:35449: connect: Connection refused (error 111)
W20260812 03:16:02.403281 8960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:02.403555 8960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:02.403596 8960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:02.407857 8960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:16:02.407917 8960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:02.407935 8960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:02.407953 8960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:16:02.408004 8960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:16:02.413471 8960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:38227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:39911,127.16.217.125:38267,127.16.217.124:35449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.124.keytab
--principal=kudu/127.16.217.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.124:35449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.217.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:02.414740 8960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:02.415995 8960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:02.425575 8970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:02.425849 8960 server_base.cc:1034] running on GCE node
W20260812 03:16:02.425576 8976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:02.425575 8972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:02.426723 8960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:02.428236 8960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:02.429435 8960 hybrid_clock.cc:648] HybridClock initialized: now 1786504562429421 us; error 29 us; skew 500 ppm
Aug 12 03:16:02 dist-test-slave-7nm7 krb5kdc[8695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504562, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:02.432394 8960 init.cc:377] Logged in from keytab as kudu/127.16.217.124@KRBTEST.COM (short username kudu)
I20260812 03:16:02.433373 8960 webserver.cc:460] Webserver started at http://127.16.217.124:43755/ using document root <none> and password file <none>
I20260812 03:16:02.433931 8960 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:02.433992 8960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:02.434180 8960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:16:02.436439 8960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-2/data/instance:
uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b"
format_stamp: "Formatted at 2026-08-12 03:16:02 on dist-test-slave-7nm7"
server_key: "dc3118b9e64a73c3dd91415655eab130"
server_key_iv: "fe38e5613ff43c58f133ff8dc79509a2"
server_key_version: "encryptionkey@0"
I20260812 03:16:02.436975 8960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-2/wal/instance:
uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b"
format_stamp: "Formatted at 2026-08-12 03:16:02 on dist-test-slave-7nm7"
server_key: "dc3118b9e64a73c3dd91415655eab130"
server_key_iv: "fe38e5613ff43c58f133ff8dc79509a2"
server_key_version: "encryptionkey@0"
I20260812 03:16:02.440304 8825 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } attempt: 1
I20260812 03:16:02.441740 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.440906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35449 (local address 127.0.0.1:51690)
0812 03:16:02.441407 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.441597 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225}
I20260812 03:16:02.442488 8960 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260812 03:16:02.443816 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.442591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35449 (local address 127.0.0.1:51700)
0812 03:16:02.443657 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.443734 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260812 03:16:02.444036 8825 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.124:35449: Network error: Client connection negotiation failed: client connection to 127.16.217.124:35449: connect: Connection refused (error 111)
I20260812 03:16:02.445763 8984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:02.447446 8960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:16:02.447616 8960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-2/wal
uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b"
format_stamp: "Formatted at 2026-08-12 03:16:02 on dist-test-slave-7nm7"
server_key: "dc3118b9e64a73c3dd91415655eab130"
server_key_iv: "fe38e5613ff43c58f133ff8dc79509a2"
server_key_version: "encryptionkey@0"
I20260812 03:16:02.447762 8960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:02.469898 8960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:02.473403 8960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:02.473635 8960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:02.481693 8960 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.124:35449
I20260812 03:16:02.482016 9076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.124:35449 every 8 connection(s)
I20260812 03:16:02.482818 8960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-2/data/info.pb
I20260812 03:16:02.485965 9077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:16:02.487406 8953 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } attempt: 3
I20260812 03:16:02.489419 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 8960
I20260812 03:16:02.489645 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504382969880-17253-0/minicluster-data/master-2/wal/instance
I20260812 03:16:02.489992 17253 external_mini_cluster.cc:1428] Setting key f61b3293cc6059e9f7bb6b7c7fc09b1a
I20260812 03:16:02.491300 9077 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:16:02 dist-test-slave-7nm7 krb5kdc[8695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504562, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.125@KRBTEST.COM for kudu/127.16.217.124@KRBTEST.COM
Aug 12 03:16:02 dist-test-slave-7nm7 krb5kdc[8695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.124@KRBTEST.COM
Aug 12 03:16:02 dist-test-slave-7nm7 krb5kdc[8695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504562, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.124@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:16:02.504247 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.488013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35449 (local address 127.0.0.1:51714)
0812 03:16:02.488228 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.488248 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:02.488353 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:02.491488 (+ 3135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:02.491495 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:02.491715 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:02.491857 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:02.491866 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.493558 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.493562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:02.494390 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:02.494401 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.495573 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.495577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:02.495659 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.496308 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:02.496328 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:02.498022 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:02.501226 (+ 3204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:02.501229 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:02.501231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:02.501545 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:02.502138 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:02.502142 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:02.502144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:02.502421 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:02.503641 (+ 1220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:02.503647 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:02.503761 (+ 114us) client_negotiation.cc:769] Sending connection context
0812 03:16:02.503881 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 03:16:02.504033 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 03:16:02.504658 8831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.493433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36912 (local address 127.16.217.126:39911)
0812 03:16:02.493572 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:02.493576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:02.493944 (+ 368us) server_negotiation.cc:407] Connection header received
0812 03:16:02.494164 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:02.494168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:02.494219 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:02.494325 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:02.495031 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.495545 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.496716 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.497062 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.497184 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.500060 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:02.500064 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:02.500067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:02.500097 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:02.502512 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.503029 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.503033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.503035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.503129 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.503395 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.503400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.503401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.503624 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:02.503723 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:02.504226 (+ 503us) server_negotiation.cc:299] Negotiation successful
0812 03:16:02.504499 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:16:02.505774 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.493326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39911 (local address 127.0.0.1:36912)
0812 03:16:02.493819 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.493872 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:02.494030 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:02.494391 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:02.494397 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:02.494778 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:02.494894 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:02.494901 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.495697 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.495700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:02.496546 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:02.496567 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.497229 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.497233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:02.497346 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.497873 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:02.497891 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:02.499927 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:02.502633 (+ 2706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:02.502646 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:02.502654 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:02.502926 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:02.503209 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:02.503214 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:02.503216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:02.503297 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:02.503794 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:02.503802 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:02.503967 (+ 165us) client_negotiation.cc:769] Sending connection context
0812 03:16:02.504896 (+ 929us) client_negotiation.cc:240] Negotiation successful
0812 03:16:02.505108 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":108,"threads_started":1}
I20260812 03:16:02.505915 9082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.490681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51722 (local address 127.16.217.124:35449)
0812 03:16:02.490995 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:02.491005 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:02.491197 (+ 192us) server_negotiation.cc:407] Connection header received
0812 03:16:02.491359 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:02.491362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:02.491404 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:02.491544 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:02.492663 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.493404 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.494582 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.495425 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.495594 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.499443 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:02.499446 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:02.499448 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:02.499494 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:02.502507 (+ 3013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.502965 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.502969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.502970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.503066 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.503757 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.503763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.503765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.504141 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:02.504550 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:02.505559 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0812 03:16:02.505706 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1012,"server-negotiator.queue_time_us":232,"spinlock_wait_cycles":23168,"thread_start_us":127,"threads_started":1}
I20260812 03:16:02.505779 9080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.488952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51714 (local address 127.16.217.124:35449)
0812 03:16:02.489653 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:02.489676 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:02.489728 (+ 52us) server_negotiation.cc:407] Connection header received
0812 03:16:02.490625 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:02.490652 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:02.491099 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:02.491545 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:02.492042 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.493404 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.494497 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.495425 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.495936 (+ 511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.498190 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:02.498199 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:02.498213 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:02.498264 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:02.501051 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.501709 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.501717 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.501723 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.501875 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.502591 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.502594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.502596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.503252 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:02.503733 (+ 481us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:02.504055 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 03:16:02.504791 (+ 736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":88,"server-negotiator.queue_time_us":521,"spinlock_wait_cycles":2176,"thread_start_us":149,"threads_started":1}
I20260812 03:16:02.507850 9077 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:16:02.510288 8953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4: Bootstrap starting.
I20260812 03:16:02.511214 8825 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } attempt: 2
Aug 12 03:16:02 dist-test-slave-7nm7 krb5kdc[8695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504562, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.124@KRBTEST.COM for kudu/127.16.217.125@KRBTEST.COM
I20260812 03:16:02.513259 8953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4: Neither blocks nor log segments found. Creating new log.
I20260812 03:16:02.514196 8953 log.cc:824] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:16:02 dist-test-slave-7nm7 krb5kdc[8695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504561, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for kudu/127.16.217.124@KRBTEST.COM
I20260812 03:16:02.517019 8953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4: No bootstrap required, opened a new log
I20260812 03:16:02.517477 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.508503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:38267 (local address 127.0.0.1:48382)
0812 03:16:02.508602 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.508618 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:02.508695 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:02.508947 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:02.508950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:02.509163 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:02.509257 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:02.509263 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.509987 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.509992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:02.510677 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:02.510687 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.511162 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.511166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:02.511264 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.511806 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:02.511821 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:02.513433 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:02.515442 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:02.515448 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:02.515455 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:02.515717 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:02.516029 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:02.516032 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:02.516034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:02.516143 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:02.516699 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:02.516702 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:02.516810 (+ 108us) client_negotiation.cc:769] Sending connection context
0812 03:16:02.516909 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 03:16:02.517036 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 03:16:02.518198 8959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.508612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48382 (local address 127.16.217.125:38267)
0812 03:16:02.508742 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:02.508747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:02.508767 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:16:02.508800 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:02.508802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:02.508845 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:02.508931 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:02.509378 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.509875 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.510821 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.511041 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.511159 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.513538 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:02.513542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:02.513545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:02.513568 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:02.515311 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.515829 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.515832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.515834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.515924 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.516263 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.516269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.516274 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.516552 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:02.516650 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:02.517914 (+ 1264us) server_negotiation.cc:299] Negotiation successful
0812 03:16:02.518030 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:16:02.518887 9077 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:16:02.521723 8953 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } }
I20260812 03:16:02.521931 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.511619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35449 (local address 127.0.0.1:51728)
0812 03:16:02.511801 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.511816 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:02.511893 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:02.512155 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:02.512158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:02.512336 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:02.512432 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:02.512439 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.513442 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.513446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:02.514274 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:02.514286 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.514940 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.514945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:02.515053 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.515589 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:02.515605 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:02.517116 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:02.520355 (+ 3239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:02.520359 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:02.520361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:02.520615 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:02.520949 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:02.520954 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:02.520956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:02.521042 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:02.521472 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:02.521475 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:02.521563 (+ 88us) client_negotiation.cc:769] Sending connection context
0812 03:16:02.521642 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 03:16:02.521787 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260812 03:16:02.521943 8953 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:16:02.522130 8953 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af0c8a29291422bb2614eceb32fdda4, State: Initialized, Role: FOLLOWER
I20260812 03:16:02.522612 9080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.511831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51728 (local address 127.16.217.124:35449)
0812 03:16:02.511940 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:02.511946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:02.511960 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:16:02.511995 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:02.511998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:02.512037 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:02.512122 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:02.512568 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.513334 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.514455 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.514798 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.515260 (+ 462us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.517290 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:02.517294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:02.517296 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:02.517324 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:02.519711 (+ 2387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.520757 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.520760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.520762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.520842 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.521156 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.521159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.521160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.521354 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:02.522290 (+ 936us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:02.522369 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 03:16:02.522482 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:16:02.522751 8953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } }
Aug 12 03:16:02 dist-test-slave-7nm7 krb5kdc[8695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:16:02.523869 9087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } } }
I20260812 03:16:02.524024 9087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [sys.catalog]: This master's current role is: FOLLOWER
Aug 12 03:16:02 dist-test-slave-7nm7 krb5kdc[8695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504562, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.124@KRBTEST.COM for kudu/127.16.217.124@KRBTEST.COM
I20260812 03:16:02.524163 8953 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:16:02.527563 8825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3: Bootstrap starting.
I20260812 03:16:02.528496 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.519420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:35449 (local address 127.0.0.1:51742)
0812 03:16:02.519553 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:02.519566 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:02.519653 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:02.519870 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:02.519873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:02.520031 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:02.520088 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:02.520093 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.521017 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.521020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:02.521912 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:02.521927 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.522637 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.522642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:02.522733 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.523160 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:02.523177 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:02.524553 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:02.526852 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:02.526855 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:02.526857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:02.527077 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:02.527358 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:02.527360 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:02.527362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:02.527442 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:02.527885 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:02.527888 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:02.527974 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 03:16:02.528172 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 03:16:02.528338 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260812 03:16:02.528942 8831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.515414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36914 (local address 127.16.217.126:39911)
0812 03:16:02.515528 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:02.515532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:02.515647 (+ 115us) server_negotiation.cc:407] Connection header received
0812 03:16:02.515829 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:02.515832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:02.515879 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:02.515933 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:02.516828 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.518014 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.520292 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.520616 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.520717 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.524735 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:02.524738 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:02.524741 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:02.524765 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:02.526651 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.527185 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.527189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.527191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.527267 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.527557 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.527560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.527562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.527766 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:02.527945 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:02.528695 (+ 750us) server_negotiation.cc:299] Negotiation successful
0812 03:16:02.528808 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:16:02.528963 9082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.519461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51742 (local address 127.16.217.124:35449)
0812 03:16:02.519554 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:02.519559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:02.519607 (+ 48us) server_negotiation.cc:407] Connection header received
0812 03:16:02.519733 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:02.519736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:02.519770 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:02.519860 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:02.520204 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.520907 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.522067 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.522509 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.522923 (+ 414us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.524695 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:02.524699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:02.524701 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:02.524737 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:02.526746 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.527177 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.527179 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.527181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.527258 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.527544 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.527547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.527548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.527756 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:02.527850 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:02.528694 (+ 844us) server_negotiation.cc:299] Negotiation successful
0812 03:16:02.528804 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 03:16:02.531081 9104 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:16:02.531169 9104 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:16:02.532435 8825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3: Neither blocks nor log segments found. Creating new log.
I20260812 03:16:02.533033 9077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b: Bootstrap starting.
I20260812 03:16:02.533522 8825 log.cc:824] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:02.529865 8834 master_runner.cc:186] Error getting master registration for 127.16.217.126:39911: OK, instance_id { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" instance_seqno: 1786504561972525 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:16:02.534020 8834 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:16:02.535989 9077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b: Neither blocks nor log segments found. Creating new log.
I20260812 03:16:02.536188 8825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3: No bootstrap required, opened a new log
I20260812 03:16:02.536811 9077 log.cc:824] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:02.538991 9077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b: No bootstrap required, opened a new log
Aug 12 03:16:02 dist-test-slave-7nm7 krb5kdc[8695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.125@KRBTEST.COM
I20260812 03:16:02.541234 8825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } }
I20260812 03:16:02.541929 8825 raft_consensus.cc:374] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:16:02.542754 8825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 649df843964f44cf80ffc1e17abaf7c3, State: Initialized, Role: FOLLOWER
I20260812 03:16:02.543610 9077 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } }
I20260812 03:16:02.543694 8825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } }
I20260812 03:16:02.543876 9077 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:16:02.543946 9077 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f5abc17fcd74ea19c8e7a30bf57e97b, State: Initialized, Role: FOLLOWER
I20260812 03:16:02.544449 9077 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } }
I20260812 03:16:02.545609 8825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:16:02.546000 9077 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:16:02.546160 8959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.534398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48396 (local address 127.16.217.125:38267)
0812 03:16:02.534513 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:02.534519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:02.534798 (+ 279us) server_negotiation.cc:407] Connection header received
0812 03:16:02.534841 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:02.534844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:02.534891 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:02.534960 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:02.535844 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.536624 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.537725 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.538036 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.538134 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.541265 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:02.541270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:02.541272 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:02.541301 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:02.543265 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.543790 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.543794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.543795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.543892 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.544224 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.544227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.544229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.544464 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:02.545183 (+ 719us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:02.545867 (+ 684us) server_negotiation.cc:299] Negotiation successful
0812 03:16:02.545992 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 03:16:02.549768 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:16:02.550464 9111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } } }
I20260812 03:16:02.550617 9111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 03:16:02.552115 9137 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:16:02.552204 9137 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:16:02.550225 8960 master_runner.cc:186] Error getting master registration for 127.16.217.126:39911: OK, instance_id { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" instance_seqno: 1786504561972525 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:16:02.553606 8960 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:16:02.558435 9113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } } }
I20260812 03:16:02.558605 9113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:16:02.562613 8959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.552277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48398 (local address 127.16.217.125:38267)
0812 03:16:02.552388 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:02.552393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:02.552811 (+ 418us) server_negotiation.cc:407] Connection header received
0812 03:16:02.553013 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:02.553017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:02.553070 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:02.553126 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:02.554168 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.555034 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.556258 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.556598 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.556715 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.558626 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:02.558630 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:02.558633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:02.558677 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:02.560712 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.561220 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.561223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.561225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.561311 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.561610 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.561614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.561616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.561844 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:02.561949 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:02.562331 (+ 382us) server_negotiation.cc:299] Negotiation successful
0812 03:16:02.562451 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 03:16:02.563814 8719 master_runner.cc:263] No leader master found from master 649df843964f44cf80ffc1e17abaf7c3
I20260812 03:16:02.563887 8719 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:16:02.568818 9082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.554012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51744 (local address 127.16.217.124:35449)
0812 03:16:02.554126 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:02.554130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:02.554290 (+ 160us) server_negotiation.cc:407] Connection header received
0812 03:16:02.554461 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:02.554464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:02.554511 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:02.554558 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:02.555469 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.556198 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.557842 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.558206 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.558307 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.564958 (+ 6651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:02.564961 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:02.564964 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:02.564992 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:02.566824 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.567344 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.567348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.567350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.567442 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.567787 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.567791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.567793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.568015 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:02.568125 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:02.568500 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 03:16:02.568644 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 03:16:02.570410 9174 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:16:02.570506 9174 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:16:02.571727 8831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:02.553532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36916 (local address 127.16.217.126:39911)
0812 03:16:02.553658 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:02.553665 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:02.557761 (+ 4096us) server_negotiation.cc:407] Connection header received
0812 03:16:02.558935 (+ 1174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:02.558938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:02.558985 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:02.559036 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:02.562833 (+ 3797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.563551 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.564888 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:02.565220 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:02.565342 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:02.567533 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:02.567537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:02.567539 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:02.567566 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:02.569257 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.570027 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.570031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.570033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.570136 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:02.570528 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:02.570532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:02.570534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:02.570803 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:02.570965 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:02.571400 (+ 435us) server_negotiation.cc:299] Negotiation successful
0812 03:16:02.571528 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 03:16:02.717927 8867 tablet.cc:2369] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:16:02.718726 8752 tablet.cc:2369] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:16:02.725152 8996 tablet.cc:2369] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:16:02.928577 9087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:16:02.928798 9087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } }
I20260812 03:16:02.929787 9087 leader_election.cc:288] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 649df843964f44cf80ffc1e17abaf7c3 (127.16.217.126:39911), 3f5abc17fcd74ea19c8e7a30bf57e97b (127.16.217.124:35449)
I20260812 03:16:02.930464 8783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4af0c8a29291422bb2614eceb32fdda4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "649df843964f44cf80ffc1e17abaf7c3" is_pre_election: true
I20260812 03:16:02.930459 9039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4af0c8a29291422bb2614eceb32fdda4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" is_pre_election: true
I20260812 03:16:02.930789 9039 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4af0c8a29291422bb2614eceb32fdda4 in term 0.
I20260812 03:16:02.930787 8783 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4af0c8a29291422bb2614eceb32fdda4 in term 0.
I20260812 03:16:02.931262 8864 leader_election.cc:302] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f5abc17fcd74ea19c8e7a30bf57e97b, 4af0c8a29291422bb2614eceb32fdda4; no voters:
I20260812 03:16:02.931564 9087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:16:02.931660 9087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:16:02.931708 9087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:16:02.932785 9087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } }
I20260812 03:16:02.933156 9087 leader_election.cc:288] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [CANDIDATE]: Term 1 election: Requested vote from peers 649df843964f44cf80ffc1e17abaf7c3 (127.16.217.126:39911), 3f5abc17fcd74ea19c8e7a30bf57e97b (127.16.217.124:35449)
I20260812 03:16:02.933542 9039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4af0c8a29291422bb2614eceb32fdda4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b"
I20260812 03:16:02.933542 8783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4af0c8a29291422bb2614eceb32fdda4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "649df843964f44cf80ffc1e17abaf7c3"
I20260812 03:16:02.933691 8783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:16:02.933691 9039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:16:02.935045 9039 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af0c8a29291422bb2614eceb32fdda4 in term 1.
I20260812 03:16:02.935045 8783 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af0c8a29291422bb2614eceb32fdda4 in term 1.
I20260812 03:16:02.935344 8864 leader_election.cc:302] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f5abc17fcd74ea19c8e7a30bf57e97b, 4af0c8a29291422bb2614eceb32fdda4; no voters:
I20260812 03:16:02.935635 9087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:16:02.935851 9087 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [term 1 LEADER]: Becoming Leader. State: Replica: 4af0c8a29291422bb2614eceb32fdda4, State: Running, Role: LEADER
I20260812 03:16:02.936133 9087 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } }
I20260812 03:16:02.938087 9176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4af0c8a29291422bb2614eceb32fdda4. Latest consensus state: current_term: 1 leader_uuid: "4af0c8a29291422bb2614eceb32fdda4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } } }
I20260812 03:16:02.938230 9176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [sys.catalog]: This master's current role is: LEADER
I20260812 03:16:02.938606 9179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:16:02.939497 9179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:16:02.944257 8783 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3 [term 1 FOLLOWER]: Refusing update from remote peer 4af0c8a29291422bb2614eceb32fdda4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:16:02.944293 9039 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b [term 1 FOLLOWER]: Refusing update from remote peer 4af0c8a29291422bb2614eceb32fdda4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:16:02.945039 9176 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:16:02.945236 9087 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:16:02.949261 9111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4af0c8a29291422bb2614eceb32fdda4. Latest consensus state: current_term: 1 leader_uuid: "4af0c8a29291422bb2614eceb32fdda4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } } }
I20260812 03:16:02.949393 9111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:16:02.949479 9113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4af0c8a29291422bb2614eceb32fdda4. Latest consensus state: current_term: 1 leader_uuid: "4af0c8a29291422bb2614eceb32fdda4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } } }
I20260812 03:16:02.949620 9113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:16:02.952358 9176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4af0c8a29291422bb2614eceb32fdda4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } } }
I20260812 03:16:02.952567 9176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [sys.catalog]: This master's current role is: LEADER
I20260812 03:16:02.952798 9176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4af0c8a29291422bb2614eceb32fdda4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } } }
I20260812 03:16:02.952864 9113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4af0c8a29291422bb2614eceb32fdda4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } } }
I20260812 03:16:02.952978 9113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:16:02.953225 9186 mvcc.cc:205] Tried to move back new op lower bound from 7317522689812701184 to 7317522689789267968. Current Snapshot: MvccSnapshot[applied={T|T < 7317522689812701184}]
I20260812 03:16:02.953970 9111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4af0c8a29291422bb2614eceb32fdda4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649df843964f44cf80ffc1e17abaf7c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39911 } } peers { permanent_uuid: "4af0c8a29291422bb2614eceb32fdda4" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 38267 } } peers { permanent_uuid: "3f5abc17fcd74ea19c8e7a30bf57e97b" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 35449 } } }
I20260812 03:16:02.954085 9111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:16:02.953033 9176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4 [sys.catalog]: This master's current role is: LEADER
I20260812 03:16:02.954442 9187 mvcc.cc:205] Tried to move back new op lower bound from 7317522689812701184 to 7317522689789267968. Current Snapshot: MvccSnapshot[applied={T|T < 7317522689812701184}]
I20260812 03:16:02.954701 9179 catalog_manager.cc:1347] Generated new cluster ID: 8f20baf0d86e47af8b0971da7343845d
I20260812 03:16:02.954769 9179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:16:02.982402 9179 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:16:02.983963 9179 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:16:02.994206 9179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4af0c8a29291422bb2614eceb32fdda4: Generated new TSK 0
I20260812 03:16:02.995012 9179 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:16:03.553669 9137 catalog_manager.cc:1259] Loaded cluster ID: 8f20baf0d86e47af8b0971da7343845d
I20260812 03:16:03.553766 9137 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b: loading cluster ID for follower catalog manager: success
I20260812 03:16:03.555725 9137 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b: acquiring CA information for follower catalog manager: success
I20260812 03:16:03.557320 9137 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3f5abc17fcd74ea19c8e7a30bf57e97b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:16:03.571913 9174 catalog_manager.cc:1259] Loaded cluster ID: 8f20baf0d86e47af8b0971da7343845d
I20260812 03:16:03.572001 9174 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3: loading cluster ID for follower catalog manager: success
I20260812 03:16:03.573618 9174 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3: acquiring CA information for follower catalog manager: success
I20260812 03:16:03.575179 9174 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 649df843964f44cf80ffc1e17abaf7c3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:16:04.073134 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 8719
I20260812 03:16:04.090090 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 8834
I20260812 03:16:04.104941 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 8960
2026-08-12T03:16:04Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3829 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:16:04 dist-test-slave-7nm7 krb5kdc[9203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:16:04 dist-test-slave-7nm7 krb5kdc[9203](info): set up 2 sockets
Aug 12 03:16:04 dist-test-slave-7nm7 krb5kdc[9203](info): commencing operation
krb5kdc: starting...
W20260812 03:16:05.443128 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:16:05 dist-test-slave-7nm7 krb5kdc[9203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:16:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:16:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:16:05.580581 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:38905
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:40475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:38905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:16:05.694790 9223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:05.695142 9223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:05.695214 9223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:05.700313 9223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:16:05.700423 9223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:05.700464 9223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:05.700513 9223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:16:05.700563 9223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:16:05.706878 9223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:40475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:38905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:38905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:05.708714 9223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:05.709903 9223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:05.719724 9235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:05.719724 9231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:05.719769 9223 server_base.cc:1034] running on GCE node
W20260812 03:16:05.719724 9232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:05.720708 9223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:05.721879 9223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:05.723095 9223 hybrid_clock.cc:648] HybridClock initialized: now 1786504565723066 us; error 53 us; skew 500 ppm
Aug 12 03:16:05 dist-test-slave-7nm7 krb5kdc[9203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504565, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:05.731974 9223 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:16:05.733136 9223 webserver.cc:460] Webserver started at http://127.16.217.126:34243/ using document root <none> and password file <none>
I20260812 03:16:05.733768 9223 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:05.733868 9223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:05.734122 9223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:16:05.737177 9223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "a92d57d6f50746b7a329c7024e7f2313"
format_stamp: "Formatted at 2026-08-12 03:16:05 on dist-test-slave-7nm7"
server_key: "0c511d22f2d834becd13567487b31d78"
server_key_iv: "44ad6b29466109984445d575619be166"
server_key_version: "encryptionkey@0"
I20260812 03:16:05.737949 9223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "a92d57d6f50746b7a329c7024e7f2313"
format_stamp: "Formatted at 2026-08-12 03:16:05 on dist-test-slave-7nm7"
server_key: "0c511d22f2d834becd13567487b31d78"
server_key_iv: "44ad6b29466109984445d575619be166"
server_key_version: "encryptionkey@0"
I20260812 03:16:05.742120 9223 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:16:05.745532 9247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:05.747156 9223 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:16:05.747423 9223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "a92d57d6f50746b7a329c7024e7f2313"
format_stamp: "Formatted at 2026-08-12 03:16:05 on dist-test-slave-7nm7"
server_key: "0c511d22f2d834becd13567487b31d78"
server_key_iv: "44ad6b29466109984445d575619be166"
server_key_version: "encryptionkey@0"
I20260812 03:16:05.747586 9223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:05.766955 9223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:05.770922 9223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:05.771162 9223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:05.779790 9223 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:38905
I20260812 03:16:05.779850 9336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:38905 every 8 connection(s)
I20260812 03:16:05.780864 9223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:16:05.785161 9337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:16:05.788751 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 9223
I20260812 03:16:05.788909 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:16:05.789517 17253 external_mini_cluster.cc:1428] Setting key 267b3708d8f21e94e7397c5ead993752
I20260812 03:16:05.792097 9337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313: Bootstrap starting.
I20260812 03:16:05.794869 9337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313: Neither blocks nor log segments found. Creating new log.
I20260812 03:16:05.795854 9337 log.cc:824] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:05.798334 9337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313: No bootstrap required, opened a new log
Aug 12 03:16:05 dist-test-slave-7nm7 krb5kdc[9203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:16:05.801673 9337 raft_consensus.cc:348] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a92d57d6f50746b7a329c7024e7f2313" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 38905 } }
I20260812 03:16:05.801999 9337 raft_consensus.cc:374] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:16:05.802093 9337 raft_consensus.cc:729] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a92d57d6f50746b7a329c7024e7f2313, State: Initialized, Role: FOLLOWER
I20260812 03:16:05.802619 9337 consensus_queue.cc:260] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a92d57d6f50746b7a329c7024e7f2313" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 38905 } }
I20260812 03:16:05.802786 9337 raft_consensus.cc:388] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:16:05.802876 9337 raft_consensus.cc:482] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:16:05.802994 9337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:16:05.804631 9337 raft_consensus.cc:504] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a92d57d6f50746b7a329c7024e7f2313" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 38905 } }
I20260812 03:16:05.805478 9337 leader_election.cc:302] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a92d57d6f50746b7a329c7024e7f2313; no voters:
I20260812 03:16:05.806069 9337 leader_election.cc:288] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:16:05.806200 9356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:16:05.806514 9356 raft_consensus.cc:686] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313 [term 1 LEADER]: Becoming Leader. State: Replica: a92d57d6f50746b7a329c7024e7f2313, State: Running, Role: LEADER
I20260812 03:16:05.806946 9356 consensus_queue.cc:237] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a92d57d6f50746b7a329c7024e7f2313" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 38905 } }
I20260812 03:16:05.807561 9342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:05.791015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41904 (local address 127.16.217.126:38905)
0812 03:16:05.791485 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:05.791501 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:05.791540 (+ 39us) server_negotiation.cc:407] Connection header received
0812 03:16:05.792172 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:05.792188 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:05.792564 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:05.792845 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:05.793825 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:05.794945 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:05.796159 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:05.796721 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:05.796917 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:05.799525 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:05.799536 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:05.799552 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:05.799581 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:05.803469 (+ 3888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:05.804091 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:05.804097 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:05.804104 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:05.804340 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:05.804698 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:05.804702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:05.804704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:05.805402 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:05.805666 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:05.805947 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 03:16:05.806368 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":163,"threads_started":1}
I20260812 03:16:05.807713 9337 sys_catalog.cc:564] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:16:05.808708 9357 sys_catalog.cc:455] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a92d57d6f50746b7a329c7024e7f2313" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a92d57d6f50746b7a329c7024e7f2313" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 38905 } } }
I20260812 03:16:05.808844 9357 sys_catalog.cc:458] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313 [sys.catalog]: This master's current role is: LEADER
I20260812 03:16:05.809186 9358 sys_catalog.cc:455] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a92d57d6f50746b7a329c7024e7f2313. Latest consensus state: current_term: 1 leader_uuid: "a92d57d6f50746b7a329c7024e7f2313" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a92d57d6f50746b7a329c7024e7f2313" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 38905 } } }
I20260812 03:16:05.809266 9358 sys_catalog.cc:458] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313 [sys.catalog]: This master's current role is: LEADER
I20260812 03:16:05.809665 9369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:16:05.811094 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:16:05.813064 9369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:16:05.818317 9369 catalog_manager.cc:1347] Generated new cluster ID: be066680fb9e49299de7e87078218745
I20260812 03:16:05.818398 9369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:16:05.829883 9369 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:16:05.831032 9369 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:16:05.845772 9369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a92d57d6f50746b7a329c7024e7f2313: Generated new TSK 0
I20260812 03:16:05.846589 9369 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:16:05.868672 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 9223
2026-08-12T03:16:05Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1755 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:16:05 dist-test-slave-7nm7 krb5kdc[9399](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:16:05 dist-test-slave-7nm7 krb5kdc[9399](info): set up 2 sockets
Aug 12 03:16:05 dist-test-slave-7nm7 krb5kdc[9399](info): commencing operation
krb5kdc: starting...
W20260812 03:16:07.187036 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:16:07 dist-test-slave-7nm7 krb5kdc[9399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:16:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:16:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:16:07.329289 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:43829
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:43829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:16:07.458627 9423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:07.459034 9423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:07.459128 9423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:07.464744 9423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:16:07.464833 9423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:07.464865 9423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:07.464891 9423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:16:07.464926 9423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:16:07.476243 9423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:43829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:43829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:07.477865 9423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:07.479360 9423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:07.490890 9434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:07.490994 9432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:07.490890 9431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:07.492388 9423 server_base.cc:1034] running on GCE node
I20260812 03:16:07.493242 9423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:07.495007 9423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:07.496174 9423 hybrid_clock.cc:648] HybridClock initialized: now 1786504567496150 us; error 46 us; skew 500 ppm
Aug 12 03:16:07 dist-test-slave-7nm7 krb5kdc[9399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504567, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:07.500506 9423 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:16:07.501874 9423 webserver.cc:460] Webserver started at http://127.16.217.126:33957/ using document root <none> and password file <none>
I20260812 03:16:07.502722 9423 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:07.502828 9423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:07.503116 9423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:16:07.506570 9423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "2357121729f14e2ab7c79d86fc24e4e7"
format_stamp: "Formatted at 2026-08-12 03:16:07 on dist-test-slave-7nm7"
server_key: "9254aebe872a6a4221b293c577530f90"
server_key_iv: "d3089256b370854b054f7f2e019e107e"
server_key_version: "encryptionkey@0"
I20260812 03:16:07.507781 9423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "2357121729f14e2ab7c79d86fc24e4e7"
format_stamp: "Formatted at 2026-08-12 03:16:07 on dist-test-slave-7nm7"
server_key: "9254aebe872a6a4221b293c577530f90"
server_key_iv: "d3089256b370854b054f7f2e019e107e"
server_key_version: "encryptionkey@0"
I20260812 03:16:07.514551 9423 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 03:16:07.517153 9441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:07.518733 9423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:16:07.518863 9423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "2357121729f14e2ab7c79d86fc24e4e7"
format_stamp: "Formatted at 2026-08-12 03:16:07 on dist-test-slave-7nm7"
server_key: "9254aebe872a6a4221b293c577530f90"
server_key_iv: "d3089256b370854b054f7f2e019e107e"
server_key_version: "encryptionkey@0"
I20260812 03:16:07.519027 9423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:07.537418 9423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:07.541694 9423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:07.541944 9423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:07.549963 9423 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:43829
I20260812 03:16:07.550496 9551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:43829 every 8 connection(s)
I20260812 03:16:07.551139 9423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:16:07.554296 9552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:16:07.560052 9552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7: Bootstrap starting.
I20260812 03:16:07.560957 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 9423
I20260812 03:16:07.561158 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:16:07.561466 17253 external_mini_cluster.cc:1428] Setting key b87e8494ad0040680b98b9ef5d7925ba
I20260812 03:16:07.563221 9552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7: Neither blocks nor log segments found. Creating new log.
I20260812 03:16:07.564021 9552 log.cc:824] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:07.566075 9552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7: No bootstrap required, opened a new log
I20260812 03:16:07.569585 9552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2357121729f14e2ab7c79d86fc24e4e7" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43829 } }
I20260812 03:16:07.569809 9552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:16:07.569844 9552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2357121729f14e2ab7c79d86fc24e4e7, State: Initialized, Role: FOLLOWER
I20260812 03:16:07.570417 9552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2357121729f14e2ab7c79d86fc24e4e7" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43829 } }
I20260812 03:16:07.570547 9552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:16:07.570626 9552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:16:07.570761 9552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:16:07.571903 9552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2357121729f14e2ab7c79d86fc24e4e7" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43829 } }
I20260812 03:16:07.572271 9552 leader_election.cc:302] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2357121729f14e2ab7c79d86fc24e4e7; no voters:
I20260812 03:16:07.572618 9552 leader_election.cc:288] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:16:07.572698 9560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:16:07.572891 9560 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 1 LEADER]: Becoming Leader. State: Replica: 2357121729f14e2ab7c79d86fc24e4e7, State: Running, Role: LEADER
Aug 12 03:16:07 dist-test-slave-7nm7 krb5kdc[9399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:16:07.573186 9560 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2357121729f14e2ab7c79d86fc24e4e7" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43829 } }
I20260812 03:16:07.573912 9552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:16:07.574394 9562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2357121729f14e2ab7c79d86fc24e4e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2357121729f14e2ab7c79d86fc24e4e7" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43829 } } }
I20260812 03:16:07.574510 9562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [sys.catalog]: This master's current role is: LEADER
I20260812 03:16:07.574437 9563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2357121729f14e2ab7c79d86fc24e4e7. Latest consensus state: current_term: 1 leader_uuid: "2357121729f14e2ab7c79d86fc24e4e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2357121729f14e2ab7c79d86fc24e4e7" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43829 } } }
I20260812 03:16:07.574558 9563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [sys.catalog]: This master's current role is: LEADER
I20260812 03:16:07.574934 9566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:16:07.578356 9566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:16:07.580478 9557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:07.563009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58536 (local address 127.16.217.126:43829)
0812 03:16:07.563564 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:07.563582 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:07.563621 (+ 39us) server_negotiation.cc:407] Connection header received
0812 03:16:07.564622 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:07.564647 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:07.565072 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:07.565429 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:07.566309 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:07.567683 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:07.569378 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:07.570546 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:07.571201 (+ 655us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:07.573617 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:07.573623 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:07.573642 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:07.573673 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:07.577052 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:07.577563 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:07.577569 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:07.577582 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:07.577692 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:07.578007 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:07.578011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:07.578013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:07.578564 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:07.578749 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:07.579538 (+ 789us) server_negotiation.cc:299] Negotiation successful
0812 03:16:07.579774 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":164,"threads_started":1}
I20260812 03:16:07.585760 9566 catalog_manager.cc:1347] Generated new cluster ID: 61b72f09f1d047dc8ea9ec918dde15a8
I20260812 03:16:07.585829 9566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:16:07.598882 9566 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:16:07.599943 9566 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:16:07.605193 9566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7: Generated new TSK 0
I20260812 03:16:07.606014 9566 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
I20260812 03:16:07.641371 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:43829
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:16:07.764739 9593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:07.765152 9593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:07.765254 9593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:07.769677 9593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:07.769738 9593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:07.769798 9593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:16:07.775846 9593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:43829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:07.777199 9593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:07.778362 9593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:07.789047 9607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:07.789145 9602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:07.789021 9604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:07.789574 9593 server_base.cc:1034] running on GCE node
I20260812 03:16:07.790555 9593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:07.791671 9593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:07.792938 9593 hybrid_clock.cc:648] HybridClock initialized: now 1786504567792898 us; error 61 us; skew 500 ppm
Aug 12 03:16:07 dist-test-slave-7nm7 krb5kdc[9399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504567, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:07.796646 9593 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:16:07.797574 9593 webserver.cc:460] Webserver started at http://127.16.217.65:45749/ using document root <none> and password file <none>
I20260812 03:16:07.798144 9593 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:07.798207 9593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:07.798456 9593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:16:07.800567 9593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5"
format_stamp: "Formatted at 2026-08-12 03:16:07 on dist-test-slave-7nm7"
server_key: "12873e39c3717afb2a1fdefeaa2a18a0"
server_key_iv: "11169fb248d24c443ef142d09561d222"
server_key_version: "encryptionkey@0"
I20260812 03:16:07.801115 9593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5"
format_stamp: "Formatted at 2026-08-12 03:16:07 on dist-test-slave-7nm7"
server_key: "12873e39c3717afb2a1fdefeaa2a18a0"
server_key_iv: "11169fb248d24c443ef142d09561d222"
server_key_version: "encryptionkey@0"
I20260812 03:16:07.805059 9593 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:16:07.807852 9618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:07.809561 9593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:16:07.809721 9593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5"
format_stamp: "Formatted at 2026-08-12 03:16:07 on dist-test-slave-7nm7"
server_key: "12873e39c3717afb2a1fdefeaa2a18a0"
server_key_iv: "11169fb248d24c443ef142d09561d222"
server_key_version: "encryptionkey@0"
I20260812 03:16:07.809855 9593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:07.839010 9593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:07.842588 9593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:07.842756 9593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:07.843333 9593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:16:07.844164 9593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:16:07.844213 9593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:07.844252 9593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:16:07.844269 9593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:07.854485 9593 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:33917
I20260812 03:16:07.854549 9816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:33917 every 8 connection(s)
I20260812 03:16:07.855643 9593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:16:07.859925 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 9593
I20260812 03:16:07.860093 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:16:07.860378 17253 external_mini_cluster.cc:1428] Setting key 38ad1413e95b50d10035f4d48000328a
Aug 12 03:16:07 dist-test-slave-7nm7 krb5kdc[9399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504567, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.217.66@KRBTEST.COM; defaulting to no policy
I20260812 03:16:07.872720 9557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:07.858408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:52981 (local address 127.16.217.126:43829)
0812 03:16:07.858575 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:07.858579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:07.859564 (+ 985us) server_negotiation.cc:407] Connection header received
0812 03:16:07.861129 (+ 1565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:07.861136 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:07.861190 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:07.861276 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:07.862800 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:07.863645 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:07.865385 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:07.865628 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:07.865714 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:07.868755 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:07.868758 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:07.868760 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:07.868786 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:07.870489 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:07.871048 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:07.871052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:07.871053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:07.871128 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:07.871533 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:07.871536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:07.871538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:07.871730 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:07.871838 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:07.872403 (+ 565us) server_negotiation.cc:299] Negotiation successful
0812 03:16:07.872537 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.66@KRBTEST.COM" created.
I20260812 03:16:07.873883 9827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:07.858873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:52981)
0812 03:16:07.859398 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:07.859444 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:07.860922 (+ 1478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:07.861489 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:07.861498 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:07.861946 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:07.862632 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:07.862648 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:07.863788 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:07.863791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:07.865237 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:07.865252 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:07.865797 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:07.865804 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:07.866032 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:07.866731 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:07.866758 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:07.868560 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:07.870623 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:07.870630 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:07.870647 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:07.870926 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:07.871243 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:07.871247 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:07.871249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:07.871425 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:07.871839 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:07.871847 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:07.872150 (+ 303us) client_negotiation.cc:769] Sending connection context
0812 03:16:07.872349 (+ 199us) client_negotiation.cc:240] Negotiation successful
0812 03:16:07.872733 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":127,"threads_started":1}
I20260812 03:16:07.875409 9824 heartbeater.cc:347] Connected to a master server at 127.16.217.126:43829
I20260812 03:16:07.875694 9824 heartbeater.cc:464] Registering TS with master...
I20260812 03:16:07.876362 9824 heartbeater.cc:511] Master 127.16.217.126:43829 requested a full tablet report, sending...
I20260812 03:16:07.878223 9472 ts_manager.cc:194] Registered new tserver with Master: 8c28bf7633604af6af8ae6a2a7d6a3f5 (127.16.217.65:33917)
I20260812 03:16:07.879981 9472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:52981
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
I20260812 03:16:07.893900 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:43829
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:16:08.012766 9831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:08.013025 9831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:08.013070 9831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:08.017179 9831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:08.017231 9831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:08.017289 9831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:16:08.022562 9831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:43829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:08.023604 9831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:08.024631 9831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:08.034502 9855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:08.034490 9851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:08.034490 9844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:08.034981 9831 server_base.cc:1034] running on GCE node
I20260812 03:16:08.035388 9831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:08.036033 9831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:08.037235 9831 hybrid_clock.cc:648] HybridClock initialized: now 1786504568037203 us; error 41 us; skew 500 ppm
Aug 12 03:16:08 dist-test-slave-7nm7 krb5kdc[9399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504568, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:08.040190 9831 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:16:08.041290 9831 webserver.cc:460] Webserver started at http://127.16.217.66:44161/ using document root <none> and password file <none>
I20260812 03:16:08.041923 9831 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:08.042021 9831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:08.042349 9831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:16:08.044901 9831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "291a4e1058664a08acc10f111160dafc"
format_stamp: "Formatted at 2026-08-12 03:16:08 on dist-test-slave-7nm7"
server_key: "fc0aa1ad542f6c7a20e1d7d2bdd99e4b"
server_key_iv: "cc7232f85f50fc6a43efa35b05a5f7fc"
server_key_version: "encryptionkey@0"
I20260812 03:16:08.045696 9831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "291a4e1058664a08acc10f111160dafc"
format_stamp: "Formatted at 2026-08-12 03:16:08 on dist-test-slave-7nm7"
server_key: "fc0aa1ad542f6c7a20e1d7d2bdd99e4b"
server_key_iv: "cc7232f85f50fc6a43efa35b05a5f7fc"
server_key_version: "encryptionkey@0"
I20260812 03:16:08.049998 9831 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 03:16:08.052664 9865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:08.054013 9831 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:16:08.054160 9831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "291a4e1058664a08acc10f111160dafc"
format_stamp: "Formatted at 2026-08-12 03:16:08 on dist-test-slave-7nm7"
server_key: "fc0aa1ad542f6c7a20e1d7d2bdd99e4b"
server_key_iv: "cc7232f85f50fc6a43efa35b05a5f7fc"
server_key_version: "encryptionkey@0"
I20260812 03:16:08.054343 9831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:08.074007 9831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:08.077476 9831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:08.077703 9831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:08.078450 9831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:16:08.079344 9831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:16:08.079419 9831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:08.079493 9831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:16:08.079542 9831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:08.088934 9831 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:37295
I20260812 03:16:08.088956 10067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:37295 every 8 connection(s)
I20260812 03:16:08.089857 9831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:16:08.092760 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 9831
I20260812 03:16:08.092885 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:16:08.093223 17253 external_mini_cluster.cc:1428] Setting key d6208b877e0546500acbfdf897f3b461
Aug 12 03:16:08 dist-test-slave-7nm7 krb5kdc[9399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504568, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.217.67@KRBTEST.COM; defaulting to no policy
I20260812 03:16:08.107327 9557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:08.091810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:45089 (local address 127.16.217.126:43829)
0812 03:16:08.091954 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:08.091958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:08.092865 (+ 907us) server_negotiation.cc:407] Connection header received
0812 03:16:08.094659 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:08.094661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:08.094707 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:08.094795 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:08.096442 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.096965 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.098477 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.098871 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.099030 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:08.102736 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:08.102743 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:08.102746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:08.102866 (+ 120us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:08.104737 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:08.105344 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:08.105350 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:08.105352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:08.105456 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:08.105827 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:08.105832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:08.105835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:08.106118 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:08.106244 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:08.106968 (+ 724us) server_negotiation.cc:299] Negotiation successful
0812 03:16:08.107110 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:16:08.108104 10073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:08.092120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:45089)
0812 03:16:08.092691 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:08.092754 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:08.094405 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:08.095000 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:08.095006 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:08.095568 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:08.096270 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:08.096294 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.097181 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.097187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:08.098336 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:08.098347 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.099035 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.099037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:08.099177 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:08.099857 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:08.099874 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:08.102416 (+ 2542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:08.104891 (+ 2475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:08.104898 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:08.104915 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:08.105210 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:08.105564 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:08.105566 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:08.105568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:08.105715 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:08.106377 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:08.106390 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:08.106645 (+ 255us) client_negotiation.cc:769] Sending connection context
0812 03:16:08.106852 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 03:16:08.107166 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":119,"threads_started":1}
I20260812 03:16:08.109391 10068 heartbeater.cc:347] Connected to a master server at 127.16.217.126:43829
I20260812 03:16:08.109635 10068 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.67@KRBTEST.COM" created.
I20260812 03:16:08.110217 10068 heartbeater.cc:511] Master 127.16.217.126:43829 requested a full tablet report, sending...
I20260812 03:16:08.111501 9472 ts_manager.cc:194] Registered new tserver with Master: 291a4e1058664a08acc10f111160dafc (127.16.217.66:37295)
I20260812 03:16:08.112249 9472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:45089
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
I20260812 03:16:08.126870 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:43829
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:16:08.243572 10077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:08.243863 10077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:08.243908 10077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:08.248907 10077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:08.248988 10077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:08.249049 10077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:16:08.255048 10077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:43829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:08.256239 10077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:08.257416 10077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:08.267884 10093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:08.267904 10088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:08.268220 10077 server_base.cc:1034] running on GCE node
W20260812 03:16:08.267894 10090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:08.268886 10077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:08.269600 10077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:08.270789 10077 hybrid_clock.cc:648] HybridClock initialized: now 1786504568270775 us; error 27 us; skew 500 ppm
Aug 12 03:16:08 dist-test-slave-7nm7 krb5kdc[9399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504568, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:08.273684 10077 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:16:08.274850 10077 webserver.cc:460] Webserver started at http://127.16.217.67:41767/ using document root <none> and password file <none>
I20260812 03:16:08.275532 10077 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:08.275631 10077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:08.275864 10077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:16:08.277992 10077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "ca2e58ef291f48ae89775f5b8f360384"
format_stamp: "Formatted at 2026-08-12 03:16:08 on dist-test-slave-7nm7"
server_key: "dbbcc1c906c9b194fb11a83381f26f8b"
server_key_iv: "2e45eb4082da5bd7022325be892fd7ec"
server_key_version: "encryptionkey@0"
I20260812 03:16:08.278645 10077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "ca2e58ef291f48ae89775f5b8f360384"
format_stamp: "Formatted at 2026-08-12 03:16:08 on dist-test-slave-7nm7"
server_key: "dbbcc1c906c9b194fb11a83381f26f8b"
server_key_iv: "2e45eb4082da5bd7022325be892fd7ec"
server_key_version: "encryptionkey@0"
I20260812 03:16:08.282853 10077 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:16:08.285735 10106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:08.287513 10077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:16:08.287662 10077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "ca2e58ef291f48ae89775f5b8f360384"
format_stamp: "Formatted at 2026-08-12 03:16:08 on dist-test-slave-7nm7"
server_key: "dbbcc1c906c9b194fb11a83381f26f8b"
server_key_iv: "2e45eb4082da5bd7022325be892fd7ec"
server_key_version: "encryptionkey@0"
I20260812 03:16:08.287788 10077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:08.300061 10077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:08.303354 10077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:08.303570 10077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:08.304198 10077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:16:08.305095 10077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:16:08.305183 10077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:08.305258 10077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:16:08.305308 10077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:08.315440 10077 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:43981
I20260812 03:16:08.315459 10324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:43981 every 8 connection(s)
I20260812 03:16:08.316402 10077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:16:08.325412 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 10077
I20260812 03:16:08.325555 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:16:08.325912 17253 external_mini_cluster.cc:1428] Setting key f196ebe32ce39bbed13b8219abd845a1
Aug 12 03:16:08 dist-test-slave-7nm7 krb5kdc[9399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504568, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:16:08.332908 9557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:08.318510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:48665 (local address 127.16.217.126:43829)
0812 03:16:08.318678 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:08.318682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:08.319489 (+ 807us) server_negotiation.cc:407] Connection header received
0812 03:16:08.320930 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:08.320933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:08.320994 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:08.321078 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:08.322303 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.322919 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.324081 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.324304 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.324421 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:08.328478 (+ 4057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:08.328482 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:08.328484 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:08.328512 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:08.330678 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:08.331249 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:08.331254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:08.331256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:08.331366 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:08.331736 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:08.331741 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:08.331743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:08.331941 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:08.332056 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:08.332604 (+ 548us) server_negotiation.cc:299] Negotiation successful
0812 03:16:08.332731 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 03:16:08.333767 10327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:08.318822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:48665)
0812 03:16:08.319319 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:08.319363 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:08.320712 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:08.321256 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:08.321263 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:08.321626 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:08.322116 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:08.322131 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.323046 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.323051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:08.323948 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:08.323958 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.324467 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.324472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:08.324649 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:08.325927 (+ 1278us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:08.325987 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:08.328274 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:08.330814 (+ 2540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:08.330822 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:08.330844 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:08.331134 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:08.331469 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:08.331472 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:08.331474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:08.331633 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:08.332056 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:08.332064 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:08.332339 (+ 275us) client_negotiation.cc:769] Sending connection context
0812 03:16:08.332538 (+ 199us) client_negotiation.cc:240] Negotiation successful
0812 03:16:08.332902 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":109,"threads_started":1}
I20260812 03:16:08.335052 10325 heartbeater.cc:347] Connected to a master server at 127.16.217.126:43829
I20260812 03:16:08.335384 10325 heartbeater.cc:464] Registering TS with master...
I20260812 03:16:08.335916 10325 heartbeater.cc:511] Master 127.16.217.126:43829 requested a full tablet report, sending...
I20260812 03:16:08.337045 9472 ts_manager.cc:194] Registered new tserver with Master: ca2e58ef291f48ae89775f5b8f360384 (127.16.217.67:43981)
I20260812 03:16:08.337785 9472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:48665
I20260812 03:16:08.341856 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:16:08.352878 9557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:08.344032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58540 (local address 127.16.217.126:43829)
0812 03:16:08.344357 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:08.344364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:08.344583 (+ 219us) server_negotiation.cc:407] Connection header received
0812 03:16:08.344935 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:08.344939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:08.345004 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:08.345103 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:08.345894 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.346509 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.347361 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.347585 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.347707 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:08.349309 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:08.349312 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:08.349315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:08.349341 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:08.351036 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:08.351502 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:08.351511 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:08.351512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:08.351589 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:08.351880 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:08.351883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:08.351885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:08.352126 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:08.352326 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:08.352569 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 03:16:08.352701 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260812 03:16:08.356109 9472 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:16:08.359469 9472 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:16:08.376989 10344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:08.370613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53738 (local address 127.16.217.65:33917)
0812 03:16:08.370991 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:08.371012 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:08.371620 (+ 608us) server_negotiation.cc:407] Connection header received
0812 03:16:08.371698 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:08.371703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:08.371864 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:08.372313 (+ 449us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:08.372482 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.373347 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.375371 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.376197 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.376453 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:08.376589 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:08.376702 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 03:16:08.376820 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":67,"threads_started":1}
I20260812 03:16:08.376984 10343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:08.370000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:33917 (local address 127.0.0.1:53738)
0812 03:16:08.371079 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:08.371095 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:08.371237 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:08.372021 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:08.372024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:08.372064 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:08.372158 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:08.372165 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.373492 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.373497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:08.375017 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:08.375028 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.376345 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.376349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:08.376489 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:08.376550 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 03:16:08.376672 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 03:16:08.376815 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1018,"mutex_wait_us":149,"thread_start_us":125,"threads_started":1}
I20260812 03:16:08.377059 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:08.370639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53036 (local address 127.16.217.66:37295)
0812 03:16:08.371390 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:08.371405 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:08.371432 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:16:08.371515 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:08.371521 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:08.371732 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:08.371897 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:08.372254 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.373495 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.375773 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.376480 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.376590 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:08.376746 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:08.376829 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 03:16:08.376914 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":116,"threads_started":1}
I20260812 03:16:08.377403 10347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:08.370902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:43981 (local address 127.0.0.1:57758)
0812 03:16:08.371208 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:08.371220 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:08.371299 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:08.372244 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:08.372247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:08.372266 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:08.372345 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:08.372351 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.373756 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.373759 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:08.375203 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:08.375213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.376339 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.376344 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:08.377107 (+ 763us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:08.377144 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 03:16:08.377231 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 03:16:08.377295 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":47,"thread_start_us":93,"threads_started":1}
I20260812 03:16:08.376992 10342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:08.369973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:37295 (local address 127.0.0.1:53036)
0812 03:16:08.370698 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:08.370760 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:08.370898 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:08.371911 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:08.371917 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:08.371942 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:08.372111 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:08.372121 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.373721 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.373724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:08.375600 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:08.375610 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.376590 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.376594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:08.376702 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:08.376727 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:16:08.376803 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 03:16:08.376869 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":126,"thread_start_us":109,"threads_started":1}
I20260812 03:16:08.377712 10348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:08.371272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57758 (local address 127.16.217.67:43981)
0812 03:16:08.371642 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:08.371651 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:08.371672 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:16:08.371736 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:08.371739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:08.371921 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:08.372071 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:08.372474 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.373588 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.375336 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.376197 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.376322 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:08.376483 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:08.377404 (+ 921us) server_negotiation.cc:299] Negotiation successful
0812 03:16:08.377530 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":170,"threads_started":1}
I20260812 03:16:08.380568 10195 tablet_service.cc:1467] Processing CreateTablet for tablet 13cd22e07c8a41ce8fa64c2ba28f5f23 (DEFAULT_TABLE table=test-table [id=c226a195ee4443bc8f4295b219bdfa28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:16:08.381623 10195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13cd22e07c8a41ce8fa64c2ba28f5f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:16:08.380944 9943 tablet_service.cc:1467] Processing CreateTablet for tablet 13cd22e07c8a41ce8fa64c2ba28f5f23 (DEFAULT_TABLE table=test-table [id=c226a195ee4443bc8f4295b219bdfa28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:16:08.380941 9711 tablet_service.cc:1467] Processing CreateTablet for tablet 13cd22e07c8a41ce8fa64c2ba28f5f23 (DEFAULT_TABLE table=test-table [id=c226a195ee4443bc8f4295b219bdfa28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:16:08.382438 9711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13cd22e07c8a41ce8fa64c2ba28f5f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:16:08.382438 9943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13cd22e07c8a41ce8fa64c2ba28f5f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:16:08.388391 10350 tablet_bootstrap.cc:492] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384: Bootstrap starting.
I20260812 03:16:08.389447 10351 tablet_bootstrap.cc:492] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc: Bootstrap starting.
I20260812 03:16:08.390143 10353 tablet_bootstrap.cc:492] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Bootstrap starting.
I20260812 03:16:08.391449 10350 tablet_bootstrap.cc:654] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384: Neither blocks nor log segments found. Creating new log.
I20260812 03:16:08.392117 10351 tablet_bootstrap.cc:654] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc: Neither blocks nor log segments found. Creating new log.
I20260812 03:16:08.392282 10353 tablet_bootstrap.cc:654] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Neither blocks nor log segments found. Creating new log.
I20260812 03:16:08.392570 10350 log.cc:824] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:08.392961 10351 log.cc:824] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:08.393110 10353 log.cc:824] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:08.395082 10350 tablet_bootstrap.cc:492] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384: No bootstrap required, opened a new log
I20260812 03:16:08.395177 10351 tablet_bootstrap.cc:492] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc: No bootstrap required, opened a new log
I20260812 03:16:08.395182 10353 tablet_bootstrap.cc:492] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5: No bootstrap required, opened a new log
I20260812 03:16:08.395321 10350 ts_tablet_manager.cc:1397] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 03:16:08.395364 10353 ts_tablet_manager.cc:1397] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260812 03:16:08.395414 10351 ts_tablet_manager.cc:1397] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260812 03:16:08.398540 10350 raft_consensus.cc:348] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } } peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } }
I20260812 03:16:08.399138 10350 raft_consensus.cc:374] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:16:08.399246 10350 raft_consensus.cc:729] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca2e58ef291f48ae89775f5b8f360384, State: Initialized, Role: FOLLOWER
I20260812 03:16:08.399345 10351 raft_consensus.cc:348] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } } peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } }
I20260812 03:16:08.399634 10351 raft_consensus.cc:374] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:16:08.399863 10351 raft_consensus.cc:729] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 291a4e1058664a08acc10f111160dafc, State: Initialized, Role: FOLLOWER
I20260812 03:16:08.399796 10350 consensus_queue.cc:260] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } } peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } }
I20260812 03:16:08.399821 10353 raft_consensus.cc:348] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } } peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } }
I20260812 03:16:08.400135 10353 raft_consensus.cc:374] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:16:08.400218 10353 raft_consensus.cc:729] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c28bf7633604af6af8ae6a2a7d6a3f5, State: Initialized, Role: FOLLOWER
I20260812 03:16:08.400370 10351 consensus_queue.cc:260] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } } peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } }
I20260812 03:16:08.400691 10353 consensus_queue.cc:260] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } } peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } }
I20260812 03:16:08.400985 10350 ts_tablet_manager.cc:1428] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:16:08.401343 10351 ts_tablet_manager.cc:1428] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 03:16:08.401367 10325 heartbeater.cc:503] Master 127.16.217.126:43829 was elected leader, sending a full tablet report...
I20260812 03:16:08.401566 10353 ts_tablet_manager.cc:1428] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260812 03:16:08.401701 10068 heartbeater.cc:503] Master 127.16.217.126:43829 was elected leader, sending a full tablet report...
I20260812 03:16:08.401940 9824 heartbeater.cc:503] Master 127.16.217.126:43829 was elected leader, sending a full tablet report...
W20260812 03:16:08.567622 10326 tablet.cc:2369] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:16:08.591245 10069 tablet.cc:2369] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:16:08.607126 9825 tablet.cc:2369] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:16:08.682690 10360 raft_consensus.cc:482] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:16:08.683027 10360 raft_consensus.cc:504] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } } peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } }
I20260812 03:16:08.684387 10360 leader_election.cc:288] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c28bf7633604af6af8ae6a2a7d6a3f5 (127.16.217.65:33917), ca2e58ef291f48ae89775f5b8f360384 (127.16.217.67:43981)
I20260812 03:16:08.691476 10348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:08.684875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:35129 (local address 127.16.217.67:43981)
0812 03:16:08.685099 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:08.685106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:08.685366 (+ 260us) server_negotiation.cc:407] Connection header received
0812 03:16:08.685515 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:08.685518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:08.685572 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:08.685666 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:08.686337 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.687473 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.689945 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.690672 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.690810 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:08.690869 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:08.691244 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 03:16:08.691333 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260812 03:16:08.691476 10344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:08.684807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:51507 (local address 127.16.217.65:33917)
0812 03:16:08.685146 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:08.685152 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:08.685366 (+ 214us) server_negotiation.cc:407] Connection header received
0812 03:16:08.685514 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:08.685517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:08.685572 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:08.685678 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:08.686342 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.687473 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.689945 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.690672 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.690810 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:08.690869 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:08.691244 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 03:16:08.691333 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260812 03:16:08.691993 10362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:08.684765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:43981 (local address 127.16.217.66:35129)
0812 03:16:08.685238 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:08.685255 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:08.685390 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:08.685701 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:08.685704 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:08.685735 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:08.685962 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:08.685971 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.687654 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.687658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:08.689665 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:08.689696 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.690823 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.690828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:08.691063 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:08.691114 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 03:16:08.691678 (+ 564us) client_negotiation.cc:240] Negotiation successful
0812 03:16:08.691815 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":304,"spinlock_wait_cycles":40064,"thread_start_us":83,"threads_started":1}
I20260812 03:16:08.691998 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:08.684657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:33917 (local address 127.16.217.66:51507)
0812 03:16:08.685190 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:08.685211 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:08.685390 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:08.685701 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:08.685704 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:08.685746 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:08.685954 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:08.685994 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.687654 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.687658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:08.689665 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:08.689696 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.690823 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.690828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:08.691040 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:08.691095 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 03:16:08.691678 (+ 583us) client_negotiation.cc:240] Negotiation successful
0812 03:16:08.691815 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":295,"spinlock_wait_cycles":24320,"thread_start_us":144,"threads_started":1}
I20260812 03:16:08.692786 9744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13cd22e07c8a41ce8fa64c2ba28f5f23" candidate_uuid: "291a4e1058664a08acc10f111160dafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" is_pre_election: true
I20260812 03:16:08.692826 10236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13cd22e07c8a41ce8fa64c2ba28f5f23" candidate_uuid: "291a4e1058664a08acc10f111160dafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca2e58ef291f48ae89775f5b8f360384" is_pre_election: true
I20260812 03:16:08.693127 10236 raft_consensus.cc:2454] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 291a4e1058664a08acc10f111160dafc in term 0.
I20260812 03:16:08.693130 9744 raft_consensus.cc:2454] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 291a4e1058664a08acc10f111160dafc in term 0.
I20260812 03:16:08.693656 9868 leader_election.cc:302] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 291a4e1058664a08acc10f111160dafc, 8c28bf7633604af6af8ae6a2a7d6a3f5, ca2e58ef291f48ae89775f5b8f360384; no voters:
I20260812 03:16:08.694037 10360 raft_consensus.cc:2781] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:16:08.694137 10360 raft_consensus.cc:482] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:16:08.694185 10360 raft_consensus.cc:3037] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:16:08.695313 10360 raft_consensus.cc:504] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } } peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } }
I20260812 03:16:08.695698 10360 leader_election.cc:288] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [CANDIDATE]: Term 1 election: Requested vote from peers 8c28bf7633604af6af8ae6a2a7d6a3f5 (127.16.217.65:33917), ca2e58ef291f48ae89775f5b8f360384 (127.16.217.67:43981)
I20260812 03:16:08.696128 9744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13cd22e07c8a41ce8fa64c2ba28f5f23" candidate_uuid: "291a4e1058664a08acc10f111160dafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5"
I20260812 03:16:08.696184 10236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13cd22e07c8a41ce8fa64c2ba28f5f23" candidate_uuid: "291a4e1058664a08acc10f111160dafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca2e58ef291f48ae89775f5b8f360384"
I20260812 03:16:08.696367 10236 raft_consensus.cc:3037] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:16:08.696244 9744 raft_consensus.cc:3037] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:16:08.697466 10236 raft_consensus.cc:2454] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 291a4e1058664a08acc10f111160dafc in term 1.
I20260812 03:16:08.697487 9744 raft_consensus.cc:2454] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 291a4e1058664a08acc10f111160dafc in term 1.
I20260812 03:16:08.697811 9868 leader_election.cc:302] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 291a4e1058664a08acc10f111160dafc, 8c28bf7633604af6af8ae6a2a7d6a3f5; no voters:
I20260812 03:16:08.697988 10360 raft_consensus.cc:2781] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:16:08.698222 10360 raft_consensus.cc:686] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [term 1 LEADER]: Becoming Leader. State: Replica: 291a4e1058664a08acc10f111160dafc, State: Running, Role: LEADER
I20260812 03:16:08.698611 10360 consensus_queue.cc:237] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } } peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } }
I20260812 03:16:08.702077 9470 catalog_manager.cc:5495] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc reported cstate change: term changed from 0 to 1, leader changed from <none> to 291a4e1058664a08acc10f111160dafc (127.16.217.66). New cstate: current_term: 1 leader_uuid: "291a4e1058664a08acc10f111160dafc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } health_report { overall_health: UNKNOWN } } }
I20260812 03:16:08.744019 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:08.740132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53046 (local address 127.16.217.66:37295)
0812 03:16:08.740354 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:08.740358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:08.740374 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:16:08.740453 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:08.740455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:08.740516 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:08.740676 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:16:08.740936 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.741447 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.742428 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:08.742646 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:08.742757 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:08.743099 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:16:08.743587 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:16:08.743728 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:08.743791 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 03:16:08.743846 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260812 03:16:08.750487 10236 raft_consensus.cc:1261] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384 [term 1 FOLLOWER]: Refusing update from remote peer 291a4e1058664a08acc10f111160dafc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:16:08.750509 9744 raft_consensus.cc:1261] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Refusing update from remote peer 291a4e1058664a08acc10f111160dafc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:16:08.751233 10360 consensus_queue.cc:1035] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:16:08.751448 10371 consensus_queue.cc:1035] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:16:08.759641 10379 mvcc.cc:205] Tried to move back new op lower bound from 7317522713593872384 to 7317522713392435200. Current Snapshot: MvccSnapshot[applied={T|T < 7317522713593872384}]
I20260812 03:16:08.762369 10380 mvcc.cc:205] Tried to move back new op lower bound from 7317522713593872384 to 7317522713392435200. Current Snapshot: MvccSnapshot[applied={T|T < 7317522713593872384 or (T in {7317522713593872384})}]
I20260812 03:16:08.767066 9472 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:16:08.767238 9472 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:16:08.769811 9472 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7: Sending DeleteTablet for 3 replicas of tablet 13cd22e07c8a41ce8fa64c2ba28f5f23
I20260812 03:16:08.770684 9711 tablet_service.cc:1514] Processing DeleteTablet for tablet 13cd22e07c8a41ce8fa64c2ba28f5f23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:16:08 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:53738
I20260812 03:16:08.770706 9943 tablet_service.cc:1514] Processing DeleteTablet for tablet 13cd22e07c8a41ce8fa64c2ba28f5f23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:16:08 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:53036
I20260812 03:16:08.770956 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 9593
I20260812 03:16:08.771598 10195 tablet_service.cc:1514] Processing DeleteTablet for tablet 13cd22e07c8a41ce8fa64c2ba28f5f23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:16:08 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:57758
I20260812 03:16:08.776316 10402 tablet_replica.cc:330] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384: stopping tablet replica
I20260812 03:16:08.776690 10402 raft_consensus.cc:2229] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:16:08.776999 10402 raft_consensus.cc:2258] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:16:08.779249 10399 tablet_replica.cc:330] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc: stopping tablet replica
I20260812 03:16:08.779682 10399 raft_consensus.cc:2229] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:16:08.780133 10399 raft_consensus.cc:2258] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:16:08.780652 10402 ts_tablet_manager.cc:1905] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:16:08.781481 10399 ts_tablet_manager.cc:1905] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:16:08.784247 10399 ts_tablet_manager.cc:1918] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:16:08.784366 10402 ts_tablet_manager.cc:1918] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:16:08.784436 10399 log.cc:1196] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/13cd22e07c8a41ce8fa64c2ba28f5f23
I20260812 03:16:08.784602 10402 log.cc:1196] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/13cd22e07c8a41ce8fa64c2ba28f5f23
I20260812 03:16:08.784808 10399 ts_tablet_manager.cc:1939] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 291a4e1058664a08acc10f111160dafc: Deleting consensus metadata
I20260812 03:16:08.785053 10402 ts_tablet_manager.cc:1939] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P ca2e58ef291f48ae89775f5b8f360384: Deleting consensus metadata
I20260812 03:16:08.786331 9442 catalog_manager.cc:4841] TS 291a4e1058664a08acc10f111160dafc (127.16.217.66:37295): tablet 13cd22e07c8a41ce8fa64c2ba28f5f23 (table test-table [id=c226a195ee4443bc8f4295b219bdfa28]) successfully deleted
I20260812 03:16:08.786561 9442 catalog_manager.cc:4841] TS ca2e58ef291f48ae89775f5b8f360384 (127.16.217.67:43981): tablet 13cd22e07c8a41ce8fa64c2ba28f5f23 (table test-table [id=c226a195ee4443bc8f4295b219bdfa28]) successfully deleted
W20260812 03:16:08.786991 9443 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.217.65:33917 (error 108)
I20260812 03:16:08.787330 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 9831
I20260812 03:16:08.790932 10342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:08.790596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:33917 (local address 127.0.0.1:53740)
0812 03:16:08.790794 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:08.790856 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:33917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260812 03:16:08.791095 9443 catalog_manager.cc:4584] TS 8c28bf7633604af6af8ae6a2a7d6a3f5 (127.16.217.65:33917): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 13cd22e07c8a41ce8fa64c2ba28f5f23: Network error: Client connection negotiation failed: client connection to 127.16.217.65:33917: connect: Connection refused (error 111)
I20260812 03:16:08.800017 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 10077
I20260812 03:16:08.816285 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 9423
I20260812 03:16:08.832593 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:43829
--webserver_interface=127.16.217.126
--webserver_port=33957
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:43829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:16:08.953809 10404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:08.954093 10404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:08.954136 10404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:08.958325 10404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:16:08.958391 10404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:08.958411 10404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:08.958427 10404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:16:08.958442 10404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:16:08.965790 10404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:43829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:43829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=33957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:08.967044 10404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:08.968147 10404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:16:08.982683 10404 server_base.cc:1034] running on GCE node
W20260812 03:16:08.982697 10421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:08.982697 10416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:08.982931 10418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:08.984090 10404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:08.986099 10404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:08.987304 10404 hybrid_clock.cc:648] HybridClock initialized: now 1786504568987283 us; error 34 us; skew 500 ppm
Aug 12 03:16:08 dist-test-slave-7nm7 krb5kdc[9399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504568, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:08.990374 10404 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:16:08.991649 10404 webserver.cc:460] Webserver started at http://127.16.217.126:33957/ using document root <none> and password file <none>
I20260812 03:16:08.992328 10404 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:08.992453 10404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:08.996634 10404 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:16:08.999168 10432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:09.000743 10404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:16:09.000901 10404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "2357121729f14e2ab7c79d86fc24e4e7"
format_stamp: "Formatted at 2026-08-12 03:16:07 on dist-test-slave-7nm7"
server_key: "9254aebe872a6a4221b293c577530f90"
server_key_iv: "d3089256b370854b054f7f2e019e107e"
server_key_version: "encryptionkey@0"
I20260812 03:16:09.001567 10404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:09.019200 10404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:09.022697 10404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:09.022931 10404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:09.031164 10404 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:43829
I20260812 03:16:09.031479 10524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:43829 every 8 connection(s)
I20260812 03:16:09.032424 10404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:16:09.037093 10525 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:16:09.038910 10525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7: Bootstrap starting.
I20260812 03:16:09.040673 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 10404
I20260812 03:16:09.041155 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:33917
--local_ip_for_outbound_sockets=127.16.217.65
--tserver_master_addrs=127.16.217.126:43829
--webserver_port=45749
--webserver_interface=127.16.217.65
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:16:09.059171 10525 log.cc:824] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:09.068255 10525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:16:09.068679 10525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7: Bootstrap complete.
I20260812 03:16:09.073091 10525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2357121729f14e2ab7c79d86fc24e4e7" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43829 } }
I20260812 03:16:09.073544 10525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2357121729f14e2ab7c79d86fc24e4e7, State: Initialized, Role: FOLLOWER
I20260812 03:16:09.074059 10525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2357121729f14e2ab7c79d86fc24e4e7" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43829 } }
I20260812 03:16:09.074221 10525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:16:09.074364 10525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:16:09.074487 10525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:16:09.076537 10525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2357121729f14e2ab7c79d86fc24e4e7" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43829 } }
I20260812 03:16:09.076903 10525 leader_election.cc:302] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2357121729f14e2ab7c79d86fc24e4e7; no voters:
I20260812 03:16:09.077306 10525 leader_election.cc:288] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:16:09.077394 10532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:16:09.077594 10532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 2 LEADER]: Becoming Leader. State: Replica: 2357121729f14e2ab7c79d86fc24e4e7, State: Running, Role: LEADER
I20260812 03:16:09.077907 10532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2357121729f14e2ab7c79d86fc24e4e7" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43829 } }
I20260812 03:16:09.078692 10525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:16:09.078994 10535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2357121729f14e2ab7c79d86fc24e4e7. Latest consensus state: current_term: 2 leader_uuid: "2357121729f14e2ab7c79d86fc24e4e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2357121729f14e2ab7c79d86fc24e4e7" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43829 } } }
I20260812 03:16:09.079128 10535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [sys.catalog]: This master's current role is: LEADER
I20260812 03:16:09.079377 10533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2357121729f14e2ab7c79d86fc24e4e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2357121729f14e2ab7c79d86fc24e4e7" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43829 } } }
I20260812 03:16:09.079483 10533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [sys.catalog]: This master's current role is: LEADER
I20260812 03:16:09.079517 10537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:16:09.085587 10537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:16:09.086460 10537 catalog_manager.cc:1259] Loaded cluster ID: 61b72f09f1d047dc8ea9ec918dde15a8
I20260812 03:16:09.086548 10537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:16:09.087656 10537 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 03:16:09.090553 10537 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 03:16:09.095120 10537 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:16:09.095768 10537 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7: Loaded TSK: 0
I20260812 03:16:09.096740 10537 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:16:09.176306 10528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:09.176640 10528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:09.176733 10528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:09.181322 10528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:09.181406 10528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:09.181491 10528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:16:09.188050 10528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:33917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=45749
--tserver_master_addrs=127.16.217.126:43829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:09.189320 10528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:09.190541 10528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:09.200472 10579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:09.200469 10578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:09.200621 10528 server_base.cc:1034] running on GCE node
W20260812 03:16:09.200469 10582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:09.201264 10528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:09.201992 10528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:09.203202 10528 hybrid_clock.cc:648] HybridClock initialized: now 1786504569203177 us; error 44 us; skew 500 ppm
Aug 12 03:16:09 dist-test-slave-7nm7 krb5kdc[9399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504569, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:09.207281 10528 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:16:09.208308 10528 webserver.cc:460] Webserver started at http://127.16.217.65:45749/ using document root <none> and password file <none>
I20260812 03:16:09.208909 10528 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:09.208988 10528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:09.213368 10528 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.002s
I20260812 03:16:09.215842 10591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:09.217242 10528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:16:09.217396 10528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5"
format_stamp: "Formatted at 2026-08-12 03:16:07 on dist-test-slave-7nm7"
server_key: "12873e39c3717afb2a1fdefeaa2a18a0"
server_key_iv: "11169fb248d24c443ef142d09561d222"
server_key_version: "encryptionkey@0"
I20260812 03:16:09.218003 10528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:09.248008 10528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:09.251694 10528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:09.251935 10528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:09.252740 10528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:16:09.253984 10603 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:16:09.259256 10528 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:16:09.259356 10528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260812 03:16:09.259445 10528 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:16:09.261271 10528 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:16:09.261343 10528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 03:16:09.262009 10603 tablet_bootstrap.cc:492] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Bootstrap starting.
I20260812 03:16:09.272101 10528 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:33917
I20260812 03:16:09.272200 10800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:33917 every 8 connection(s)
I20260812 03:16:09.273098 10528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:16:09.281329 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 10528
I20260812 03:16:09.281826 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:37295
--local_ip_for_outbound_sockets=127.16.217.66
--tserver_master_addrs=127.16.217.126:43829
--webserver_port=44161
--webserver_interface=127.16.217.66
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:16:09.282739 10603 log.cc:824] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:16:09 dist-test-slave-7nm7 krb5kdc[9399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504569, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:16:09.291353 10804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:09.275210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:46099 (local address 127.16.217.126:43829)
0812 03:16:09.276365 (+ 1155us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:09.276382 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:09.276427 (+ 45us) server_negotiation.cc:407] Connection header received
0812 03:16:09.277197 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:09.277207 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:09.277476 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:09.277801 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:09.279078 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.280244 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.281533 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.282060 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.282209 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:09.285974 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:09.285979 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:09.285991 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:09.286022 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:09.287856 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:09.288469 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:09.288474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:09.288477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:09.288589 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:09.288939 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:09.288943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:09.288945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:09.289412 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:09.289580 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:09.290139 (+ 559us) server_negotiation.cc:299] Negotiation successful
0812 03:16:09.290429 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1012,"thread_start_us":113,"threads_started":1}
I20260812 03:16:09.291368 10805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:09.275263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:46099)
0812 03:16:09.275795 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:09.275849 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:09.276789 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:09.277856 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:09.277864 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:09.278380 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:09.278887 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:09.278903 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.280409 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.280413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:09.281322 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:09.281333 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.282440 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.282443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:09.282581 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:09.283231 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:09.283253 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:09.285705 (+ 2452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:09.288041 (+ 2336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:09.288047 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:09.288060 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:09.288322 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:09.288694 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:09.288697 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:09.288699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:09.288839 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:09.289565 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:09.289571 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:09.289829 (+ 258us) client_negotiation.cc:769] Sending connection context
0812 03:16:09.290020 (+ 191us) client_negotiation.cc:240] Negotiation successful
0812 03:16:09.290420 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":158,"threads_started":1}
I20260812 03:16:09.292625 10603 tablet_bootstrap.cc:492] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:16:09.293140 10603 tablet_bootstrap.cc:492] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Bootstrap complete.
I20260812 03:16:09.294166 10801 heartbeater.cc:347] Connected to a master server at 127.16.217.126:43829
I20260812 03:16:09.294510 10801 heartbeater.cc:464] Registering TS with master...
I20260812 03:16:09.294759 10603 ts_tablet_manager.cc:1397] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Time spent bootstrapping tablet: real 0.033s user 0.015s sys 0.016s
I20260812 03:16:09.295243 10801 heartbeater.cc:511] Master 127.16.217.126:43829 requested a full tablet report, sending...
I20260812 03:16:09.297686 10457 ts_manager.cc:194] Registered new tserver with Master: 8c28bf7633604af6af8ae6a2a7d6a3f5 (127.16.217.65:33917)
I20260812 03:16:09.298482 10603 raft_consensus.cc:348] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } } peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } }
I20260812 03:16:09.299036 10603 raft_consensus.cc:729] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c28bf7633604af6af8ae6a2a7d6a3f5, State: Initialized, Role: FOLLOWER
I20260812 03:16:09.299531 10603 consensus_queue.cc:260] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } } peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } }
I20260812 03:16:09.300558 10603 ts_tablet_manager.cc:1428] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260812 03:16:09.302150 10457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:46099
I20260812 03:16:09.304993 10801 heartbeater.cc:503] Master 127.16.217.126:43829 was elected leader, sending a full tablet report...
Aug 12 03:16:09 dist-test-slave-7nm7 krb5kdc[9399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504568, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for kudu/127.16.217.65@KRBTEST.COM
I20260812 03:16:09.312752 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:09.301262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53748 (local address 127.16.217.65:33917)
0812 03:16:09.301949 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:09.301960 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:09.301984 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:16:09.302055 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:09.302060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:09.302292 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:09.302441 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:09.303165 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.303918 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.304931 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.305445 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.305565 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:09.308415 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:09.308427 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:09.308444 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:09.308474 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:09.310309 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:09.310908 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:09.310914 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:09.310916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:09.311017 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:09.311480 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:09.311484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:09.311485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:09.311816 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:09.311950 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:09.312395 (+ 445us) server_negotiation.cc:299] Negotiation successful
0812 03:16:09.312574 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":91,"threads_started":1}
I20260812 03:16:09.312791 10813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:09.301164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:33917 (local address 127.0.0.1:53748)
0812 03:16:09.301603 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:09.301650 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:09.301790 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:09.302446 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:09.302452 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:09.302902 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:09.303037 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:09.303045 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.304049 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.304054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:09.304792 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:09.304802 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.305736 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.305740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:09.305847 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:09.306542 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:09.306557 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:09.308294 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:09.310427 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:09.310439 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:09.310448 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:09.310771 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:09.311128 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:09.311134 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:09.311138 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:09.311359 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:09.311953 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:09.311962 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:09.312133 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 03:16:09.312296 (+ 163us) client_negotiation.cc:240] Negotiation successful
0812 03:16:09.312531 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":135,"threads_started":1}
I20260812 03:16:09.314298 10685 tablet_service.cc:1514] Processing DeleteTablet for tablet 13cd22e07c8a41ce8fa64c2ba28f5f23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:16:08 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:53748
I20260812 03:16:09.315410 10815 tablet_replica.cc:330] stopping tablet replica
I20260812 03:16:09.315837 10815 raft_consensus.cc:2229] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:16:09.316123 10815 raft_consensus.cc:2258] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:16:09.317108 10815 ts_tablet_manager.cc:1905] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:16:09.321622 10815 ts_tablet_manager.cc:1918] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:16:09.321713 10815 log.cc:1196] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/wal/wals/13cd22e07c8a41ce8fa64c2ba28f5f23
I20260812 03:16:09.322119 10815 ts_tablet_manager.cc:1939] T 13cd22e07c8a41ce8fa64c2ba28f5f23 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Deleting consensus metadata
I20260812 03:16:09.324501 10434 catalog_manager.cc:4841] TS 8c28bf7633604af6af8ae6a2a7d6a3f5 (127.16.217.65:33917): tablet 13cd22e07c8a41ce8fa64c2ba28f5f23 (table test-table [id=c226a195ee4443bc8f4295b219bdfa28]) successfully deleted
W20260812 03:16:09.408882 10810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:09.409147 10810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:09.409201 10810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:09.413283 10810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:09.413348 10810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:09.413409 10810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:16:09.418588 10810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:37295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=44161
--tserver_master_addrs=127.16.217.126:43829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:09.419621 10810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:09.420691 10810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:09.432644 10824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:09.432722 10810 server_base.cc:1034] running on GCE node
W20260812 03:16:09.432657 10831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:09.432683 10825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:09.433357 10810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:09.434048 10810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:09.435209 10810 hybrid_clock.cc:648] HybridClock initialized: now 1786504569435184 us; error 42 us; skew 500 ppm
Aug 12 03:16:09 dist-test-slave-7nm7 krb5kdc[9399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504569, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:09.438006 10810 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:16:09.439065 10810 webserver.cc:460] Webserver started at http://127.16.217.66:44161/ using document root <none> and password file <none>
I20260812 03:16:09.439675 10810 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:09.439790 10810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:09.444103 10810 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 03:16:09.446514 10838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:09.447777 10810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:16:09.447926 10810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "291a4e1058664a08acc10f111160dafc"
format_stamp: "Formatted at 2026-08-12 03:16:08 on dist-test-slave-7nm7"
server_key: "fc0aa1ad542f6c7a20e1d7d2bdd99e4b"
server_key_iv: "cc7232f85f50fc6a43efa35b05a5f7fc"
server_key_version: "encryptionkey@0"
I20260812 03:16:09.448503 10810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:09.468214 10810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:09.471959 10810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:09.472201 10810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:09.472858 10810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:16:09.473750 10810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:16:09.473829 10810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:09.473903 10810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:16:09.473948 10810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:09.485381 10810 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:37295
I20260812 03:16:09.485404 11040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:37295 every 8 connection(s)
I20260812 03:16:09.486557 10810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:16:09.491540 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 10810
I20260812 03:16:09.492010 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:43981
--local_ip_for_outbound_sockets=127.16.217.67
--tserver_master_addrs=127.16.217.126:43829
--webserver_port=41767
--webserver_interface=127.16.217.67
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:16:09 dist-test-slave-7nm7 krb5kdc[9399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504569, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:16:09.504854 10804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:09.488760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:48105 (local address 127.16.217.126:43829)
0812 03:16:09.488932 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:09.488936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:09.489902 (+ 966us) server_negotiation.cc:407] Connection header received
0812 03:16:09.491492 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:09.491497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:09.491567 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:09.491691 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:09.493058 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.493587 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.495439 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.495718 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.495817 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:09.500002 (+ 4185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:09.500007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:09.500009 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:09.500039 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:09.501701 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:09.502867 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:09.502879 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:09.502880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:09.502962 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:09.503552 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:09.503555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:09.503557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:09.503781 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:09.503901 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:09.504478 (+ 577us) server_negotiation.cc:299] Negotiation successful
0812 03:16:09.504634 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 03:16:09.505910 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:09.489248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:48105)
0812 03:16:09.489734 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:09.489778 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:09.491260 (+ 1482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:09.491897 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:09.491904 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:09.492390 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:09.492891 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:09.492906 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.493725 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.493731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:09.495276 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:09.495292 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.495901 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.495907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:09.496146 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:09.496880 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:09.496911 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:09.499762 (+ 2851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:09.502105 (+ 2343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:09.502116 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:09.502147 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:09.502700 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:09.503225 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:09.503229 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:09.503231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:09.503424 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:09.503901 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:09.503910 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:09.504207 (+ 297us) client_negotiation.cc:769] Sending connection context
0812 03:16:09.504419 (+ 212us) client_negotiation.cc:240] Negotiation successful
0812 03:16:09.504804 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":117,"threads_started":1}
I20260812 03:16:09.507262 11041 heartbeater.cc:347] Connected to a master server at 127.16.217.126:43829
I20260812 03:16:09.507542 11041 heartbeater.cc:464] Registering TS with master...
I20260812 03:16:09.508122 11041 heartbeater.cc:511] Master 127.16.217.126:43829 requested a full tablet report, sending...
I20260812 03:16:09.509352 10457 ts_manager.cc:194] Registered new tserver with Master: 291a4e1058664a08acc10f111160dafc (127.16.217.66:37295)
I20260812 03:16:09.509970 10457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:48105
W20260812 03:16:09.611325 11045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:09.611610 11045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:09.611657 11045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:09.615916 11045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:09.615990 11045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:09.616051 11045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:16:09.621668 11045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:43981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=41767
--tserver_master_addrs=127.16.217.126:43829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:09.622982 11045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:09.624051 11045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:16:09.634917 11045 server_base.cc:1034] running on GCE node
W20260812 03:16:09.634902 11054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:09.634902 11053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:09.635150 11057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:09.635600 11045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:09.636335 11045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:09.637637 11045 hybrid_clock.cc:648] HybridClock initialized: now 1786504569637565 us; error 95 us; skew 500 ppm
Aug 12 03:16:09 dist-test-slave-7nm7 krb5kdc[9399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504569, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:09.640816 11045 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:16:09.641868 11045 webserver.cc:460] Webserver started at http://127.16.217.67:41767/ using document root <none> and password file <none>
I20260812 03:16:09.642557 11045 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:09.642704 11045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:09.646934 11045 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 03:16:09.649345 11075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:09.650698 11045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:16:09.650842 11045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "ca2e58ef291f48ae89775f5b8f360384"
format_stamp: "Formatted at 2026-08-12 03:16:08 on dist-test-slave-7nm7"
server_key: "dbbcc1c906c9b194fb11a83381f26f8b"
server_key_iv: "2e45eb4082da5bd7022325be892fd7ec"
server_key_version: "encryptionkey@0"
I20260812 03:16:09.651419 11045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:09.671233 11045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:09.674819 11045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:09.675046 11045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:09.675688 11045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:16:09.676570 11045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:16:09.676646 11045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:09.676720 11045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:16:09.676774 11045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:09.686937 11045 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:43981
I20260812 03:16:09.686983 11252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:43981 every 8 connection(s)
I20260812 03:16:09.688050 11045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:16:09.691227 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 11045
Aug 12 03:16:09 dist-test-slave-7nm7 krb5kdc[9399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504569, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:16:09.700891 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:09.691953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58556 (local address 127.16.217.126:43829)
0812 03:16:09.692469 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:09.692472 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:09.692483 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:16:09.692521 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:09.692523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:09.692561 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:09.692665 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:09.693481 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.694056 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.694974 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.695203 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.695460 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:09.697246 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:09.697250 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:09.697253 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:09.697275 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:09.698918 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:09.699533 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:09.699537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:09.699538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:09.699627 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:09.699907 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:09.699911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:09.699913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:09.700111 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:09.700342 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:09.700543 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 03:16:09.700675 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":79,"threads_started":1}
I20260812 03:16:09.702780 10804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:09.690202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:51329 (local address 127.16.217.126:43829)
0812 03:16:09.690383 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:09.690387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:09.691248 (+ 861us) server_negotiation.cc:407] Connection header received
0812 03:16:09.692705 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:09.692709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:09.692755 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:09.692851 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:09.693988 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.694531 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.695661 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.695956 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.696070 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:09.698671 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:09.698677 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:09.698683 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:09.698728 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:09.700151 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:09.700677 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:09.700683 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:09.700688 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:09.700781 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:09.701219 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:09.701224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:09.701226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:09.701674 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:09.701784 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:09.702559 (+ 775us) server_negotiation.cc:299] Negotiation successful
0812 03:16:09.702676 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 03:16:09.703718 11255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:09.690627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:51329)
0812 03:16:09.691090 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:09.691131 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:09.692500 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:09.693019 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:09.693025 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:09.693383 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:09.693849 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:09.693863 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.694659 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.694667 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:09.695523 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:09.695532 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.696173 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.696180 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:09.696403 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:09.696919 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:09.696940 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:09.698506 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:09.700290 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:09.700296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:09.700312 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:09.700575 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:09.700893 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:09.700898 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:09.700903 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:09.701110 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:09.701791 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:09.701797 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:09.702140 (+ 343us) client_negotiation.cc:769] Sending connection context
0812 03:16:09.702402 (+ 262us) client_negotiation.cc:240] Negotiation successful
0812 03:16:09.702754 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":110,"threads_started":1}
I20260812 03:16:09.704965 11253 heartbeater.cc:347] Connected to a master server at 127.16.217.126:43829
I20260812 03:16:09.705220 11253 heartbeater.cc:464] Registering TS with master...
I20260812 03:16:09.705744 11253 heartbeater.cc:511] Master 127.16.217.126:43829 requested a full tablet report, sending...
I20260812 03:16:09.706835 10457 ts_manager.cc:194] Registered new tserver with Master: ca2e58ef291f48ae89775f5b8f360384 (127.16.217.67:43981)
I20260812 03:16:09.707401 10457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:51329
I20260812 03:16:09.715880 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:16:09.720768 10804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:09.716845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58560 (local address 127.16.217.126:43829)
0812 03:16:09.716970 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:09.716974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:09.717011 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:16:09.717182 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:09.717184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:09.717292 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:09.717382 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:16:09.717674 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.718275 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.719179 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.719416 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.719507 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:09.719789 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:16:09.720223 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:16:09.720433 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:09.720495 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 03:16:09.720555 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:16:09.721916 10457 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:16:09.725425 10457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:16:09.734251 10685 tablet_service.cc:1467] Processing CreateTablet for tablet 3f4b1325c29b4875843a330e6a1058d5 (DEFAULT_TABLE table=test-table [id=c8ccc73dafc24540ae792d9b2d38aeaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:16:09.734985 10685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f4b1325c29b4875843a330e6a1058d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:16:09.738111 10603 tablet_bootstrap.cc:492] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Bootstrap starting.
I20260812 03:16:09.738808 10813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:09.733049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:43981 (local address 127.0.0.1:57766)
0812 03:16:09.733192 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:09.733215 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:09.733314 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:09.734669 (+ 1355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:09.734672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:09.734693 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:09.734892 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:09.734900 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.736012 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.736018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:09.737034 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:09.737046 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.738275 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.738280 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:09.738404 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:09.738471 (+ 67us) client_negotiation.cc:769] Sending connection context
0812 03:16:09.738535 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:16:09.738657 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":40}
I20260812 03:16:09.739112 10603 tablet_bootstrap.cc:654] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Neither blocks nor log segments found. Creating new log.
I20260812 03:16:09.739576 11275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:09.733404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57766 (local address 127.16.217.67:43981)
0812 03:16:09.733948 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:09.733962 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:09.733988 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:16:09.734068 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:09.734074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:09.734348 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:09.734523 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:09.735063 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.735870 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.737174 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.737971 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.739121 (+ 1150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:09.739241 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:09.739299 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 03:16:09.739401 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":203,"threads_started":1}
I20260812 03:16:09.739730 11274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:09.733475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:37295 (local address 127.0.0.1:53062)
0812 03:16:09.733836 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:09.733849 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:09.733942 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:09.734720 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:09.734723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:09.734742 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:09.734917 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:09.734924 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.736545 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.736549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:09.737676 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:09.737688 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.739292 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.739296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:09.739387 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:09.739411 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:16:09.739494 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 03:16:09.739582 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":25,"thread_start_us":132,"threads_started":1}
I20260812 03:16:09.740332 11276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:09.733643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53062 (local address 127.16.217.66:37295)
0812 03:16:09.734106 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:09.734115 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:09.734134 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:16:09.734198 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:09.734202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:09.734406 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:09.734570 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:09.735212 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.736262 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.737822 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.738934 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.739060 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:09.739222 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:09.739565 (+ 343us) server_negotiation.cc:299] Negotiation successful
0812 03:16:09.739989 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":128,"threads_started":1}
I20260812 03:16:09.740972 10603 tablet_bootstrap.cc:492] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5: No bootstrap required, opened a new log
I20260812 03:16:09.741083 10603 ts_tablet_manager.cc:1397] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:16:09.741564 10603 raft_consensus.cc:348] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } } peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } }
I20260812 03:16:09.741698 10603 raft_consensus.cc:374] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:16:09.741739 10603 raft_consensus.cc:729] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c28bf7633604af6af8ae6a2a7d6a3f5, State: Initialized, Role: FOLLOWER
I20260812 03:16:09.741875 10603 consensus_queue.cc:260] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } } peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } }
I20260812 03:16:09.742195 10603 ts_tablet_manager.cc:1428] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:16:09.743228 10923 tablet_service.cc:1467] Processing CreateTablet for tablet 3f4b1325c29b4875843a330e6a1058d5 (DEFAULT_TABLE table=test-table [id=c8ccc73dafc24540ae792d9b2d38aeaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:16:09.743177 11153 tablet_service.cc:1467] Processing CreateTablet for tablet 3f4b1325c29b4875843a330e6a1058d5 (DEFAULT_TABLE table=test-table [id=c8ccc73dafc24540ae792d9b2d38aeaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:16:09.744277 10923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f4b1325c29b4875843a330e6a1058d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:16:09.744313 11153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f4b1325c29b4875843a330e6a1058d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:16:09.751091 11280 tablet_bootstrap.cc:492] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384: Bootstrap starting.
I20260812 03:16:09.751538 11284 tablet_bootstrap.cc:492] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc: Bootstrap starting.
I20260812 03:16:09.753832 11280 tablet_bootstrap.cc:654] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384: Neither blocks nor log segments found. Creating new log.
I20260812 03:16:09.753863 11284 tablet_bootstrap.cc:654] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc: Neither blocks nor log segments found. Creating new log.
I20260812 03:16:09.754691 11284 log.cc:824] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:09.754715 11280 log.cc:824] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:09.756906 11284 tablet_bootstrap.cc:492] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc: No bootstrap required, opened a new log
I20260812 03:16:09.756916 11280 tablet_bootstrap.cc:492] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384: No bootstrap required, opened a new log
I20260812 03:16:09.757120 11284 ts_tablet_manager.cc:1397] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:16:09.757133 11280 ts_tablet_manager.cc:1397] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:16:09.760026 11280 raft_consensus.cc:348] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } } peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } }
I20260812 03:16:09.760025 11284 raft_consensus.cc:348] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } } peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } }
I20260812 03:16:09.760243 11280 raft_consensus.cc:374] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:16:09.760272 11284 raft_consensus.cc:374] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:16:09.760304 11280 raft_consensus.cc:729] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca2e58ef291f48ae89775f5b8f360384, State: Initialized, Role: FOLLOWER
I20260812 03:16:09.760358 11284 raft_consensus.cc:729] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 291a4e1058664a08acc10f111160dafc, State: Initialized, Role: FOLLOWER
I20260812 03:16:09.760799 11280 consensus_queue.cc:260] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } } peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } }
I20260812 03:16:09.760799 11284 consensus_queue.cc:260] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } } peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } }
I20260812 03:16:09.761530 11041 heartbeater.cc:503] Master 127.16.217.126:43829 was elected leader, sending a full tablet report...
I20260812 03:16:09.761970 11253 heartbeater.cc:503] Master 127.16.217.126:43829 was elected leader, sending a full tablet report...
I20260812 03:16:09.762100 11280 ts_tablet_manager.cc:1428] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 03:16:09.762786 11284 ts_tablet_manager.cc:1428] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
W20260812 03:16:09.774492 10803 tablet.cc:2369] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:16:09.939081 11254 tablet.cc:2369] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:16:09.987891 11043 tablet.cc:2369] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:16:09.993813 11287 raft_consensus.cc:482] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:16:09.994016 11287 raft_consensus.cc:504] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } } peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } }
I20260812 03:16:09.995682 11287 leader_election.cc:288] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca2e58ef291f48ae89775f5b8f360384 (127.16.217.67:43981), 8c28bf7633604af6af8ae6a2a7d6a3f5 (127.16.217.65:33917)
I20260812 03:16:09.999467 11275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:09.996036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:38947 (local address 127.16.217.67:43981)
0812 03:16:09.996189 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:09.996195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:09.996220 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:16:09.996291 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:09.996297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:09.996360 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:09.996454 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:09.996809 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.997351 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.998445 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.998845 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.998975 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:09.999042 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:09.999269 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 03:16:09.999342 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 03:16:09.999468 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:09.995923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:43981 (local address 127.16.217.66:38947)
0812 03:16:09.996064 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:09.996079 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:09.996183 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:09.996499 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:09.996516 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:09.996538 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:09.996690 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:09.996700 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.997471 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.997478 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:09.998321 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:09.998331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.998969 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.998972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:09.999139 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:09.999159 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 03:16:09.999249 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 03:16:09.999326 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":48}
I20260812 03:16:10.000174 11185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4b1325c29b4875843a330e6a1058d5" candidate_uuid: "291a4e1058664a08acc10f111160dafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca2e58ef291f48ae89775f5b8f360384" is_pre_election: true
I20260812 03:16:10.000437 11289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:09.996133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:33917 (local address 127.16.217.66:53037)
0812 03:16:09.996525 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:09.996539 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:09.996618 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:09.997148 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:09.997151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:09.997171 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:09.997310 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:09.997316 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.998221 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.998251 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:09.999090 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:09.999101 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.999993 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.999998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:10.000100 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:10.000125 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:16:10.000206 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 03:16:10.000281 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":1,"thread_start_us":149,"threads_started":1}
I20260812 03:16:10.000433 11290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:09.996619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:53037 (local address 127.16.217.65:33917)
0812 03:16:09.996854 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:09.996858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:09.996871 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:16:09.996920 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:09.996922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:09.996980 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:09.997125 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:09.997440 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.998092 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.999235 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:09.999805 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:09.999986 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:10.000145 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:10.000233 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 03:16:10.000307 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":79,"threads_started":1}
I20260812 03:16:10.000543 11185 raft_consensus.cc:2454] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 291a4e1058664a08acc10f111160dafc in term 0.
I20260812 03:16:10.001034 10725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4b1325c29b4875843a330e6a1058d5" candidate_uuid: "291a4e1058664a08acc10f111160dafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" is_pre_election: true
I20260812 03:16:10.001088 10841 leader_election.cc:302] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 291a4e1058664a08acc10f111160dafc, ca2e58ef291f48ae89775f5b8f360384; no voters:
I20260812 03:16:10.001314 10725 raft_consensus.cc:2454] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 291a4e1058664a08acc10f111160dafc in term 0.
I20260812 03:16:10.001377 11287 raft_consensus.cc:2781] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:16:10.001523 11287 raft_consensus.cc:482] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:16:10.001610 11287 raft_consensus.cc:3037] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:16:10.003129 11287 raft_consensus.cc:504] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } } peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } }
I20260812 03:16:10.003551 11287 leader_election.cc:288] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [CANDIDATE]: Term 1 election: Requested vote from peers ca2e58ef291f48ae89775f5b8f360384 (127.16.217.67:43981), 8c28bf7633604af6af8ae6a2a7d6a3f5 (127.16.217.65:33917)
I20260812 03:16:10.003925 11185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4b1325c29b4875843a330e6a1058d5" candidate_uuid: "291a4e1058664a08acc10f111160dafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca2e58ef291f48ae89775f5b8f360384"
I20260812 03:16:10.003933 10725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4b1325c29b4875843a330e6a1058d5" candidate_uuid: "291a4e1058664a08acc10f111160dafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5"
I20260812 03:16:10.004083 10725 raft_consensus.cc:3037] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:16:10.004070 11185 raft_consensus.cc:3037] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:16:10.005157 10725 raft_consensus.cc:2454] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 291a4e1058664a08acc10f111160dafc in term 1.
I20260812 03:16:10.005388 11185 raft_consensus.cc:2454] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 291a4e1058664a08acc10f111160dafc in term 1.
I20260812 03:16:10.005517 10843 leader_election.cc:302] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 291a4e1058664a08acc10f111160dafc, 8c28bf7633604af6af8ae6a2a7d6a3f5; no voters:
I20260812 03:16:10.005708 11287 raft_consensus.cc:2781] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:16:10.005960 11287 raft_consensus.cc:686] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [term 1 LEADER]: Becoming Leader. State: Replica: 291a4e1058664a08acc10f111160dafc, State: Running, Role: LEADER
I20260812 03:16:10.006400 11287 consensus_queue.cc:237] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } } peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } }
I20260812 03:16:10.009960 10457 catalog_manager.cc:5495] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc reported cstate change: term changed from 0 to 1, leader changed from <none> to 291a4e1058664a08acc10f111160dafc (127.16.217.66). New cstate: current_term: 1 leader_uuid: "291a4e1058664a08acc10f111160dafc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260812 03:16:10.042205 11276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:10.037947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53070 (local address 127.16.217.66:37295)
0812 03:16:10.038094 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:10.038099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:10.038119 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:16:10.038276 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:10.038279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:10.038351 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:10.038440 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:16:10.038677 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:10.039183 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:10.040376 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:10.040693 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:10.040791 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:10.041167 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:16:10.041602 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:16:10.041700 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:10.041925 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 03:16:10.042058 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:16:10.049408 11185 raft_consensus.cc:1261] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384 [term 1 FOLLOWER]: Refusing update from remote peer 291a4e1058664a08acc10f111160dafc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:16:10.049569 10725 raft_consensus.cc:1261] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Refusing update from remote peer 291a4e1058664a08acc10f111160dafc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:16:10.050601 11291 consensus_queue.cc:1035] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:16:10.050830 11287 consensus_queue.cc:1035] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:16:10.057693 11293 mvcc.cc:205] Tried to move back new op lower bound from 7317522718911094784 to 7317522718749106176. Current Snapshot: MvccSnapshot[applied={T|T < 7317522718911094784}]
I20260812 03:16:10.068756 10457 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:16:10.068967 10457 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:16:10.072224 10457 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7: Sending DeleteTablet for 3 replicas of tablet 3f4b1325c29b4875843a330e6a1058d5
I20260812 03:16:10.073558 10685 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f4b1325c29b4875843a330e6a1058d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:16:10 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:53748
I20260812 03:16:10.073689 11153 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f4b1325c29b4875843a330e6a1058d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:16:10 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:57766
I20260812 03:16:10.074075 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 10528
I20260812 03:16:10.074292 10923 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f4b1325c29b4875843a330e6a1058d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:16:10 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:53062
I20260812 03:16:10.080210 11319 tablet_replica.cc:330] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384: stopping tablet replica
I20260812 03:16:10.080686 11319 raft_consensus.cc:2229] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:16:10.081151 11319 raft_consensus.cc:2258] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:16:10.084959 11319 ts_tablet_manager.cc:1905] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:16:10.086567 11320 tablet_replica.cc:330] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc: stopping tablet replica
I20260812 03:16:10.086992 11320 raft_consensus.cc:2229] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:16:10.087477 11320 raft_consensus.cc:2258] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:16:10.088701 11319 ts_tablet_manager.cc:1918] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:16:10.088845 11319 log.cc:1196] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/3f4b1325c29b4875843a330e6a1058d5
I20260812 03:16:10.089224 11319 ts_tablet_manager.cc:1939] T 3f4b1325c29b4875843a330e6a1058d5 P ca2e58ef291f48ae89775f5b8f360384: Deleting consensus metadata
I20260812 03:16:10.089707 11320 ts_tablet_manager.cc:1905] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:16:10.091418 10433 catalog_manager.cc:4841] TS ca2e58ef291f48ae89775f5b8f360384 (127.16.217.67:43981): tablet 3f4b1325c29b4875843a330e6a1058d5 (table test-table [id=c8ccc73dafc24540ae792d9b2d38aeaa]) successfully deleted
W20260812 03:16:10.092430 10434 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.217.65:33917 (error 108)
I20260812 03:16:10.093158 11320 ts_tablet_manager.cc:1918] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:16:10.093261 11320 log.cc:1196] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/3f4b1325c29b4875843a330e6a1058d5
I20260812 03:16:10.093602 11320 ts_tablet_manager.cc:1939] T 3f4b1325c29b4875843a330e6a1058d5 P 291a4e1058664a08acc10f111160dafc: Deleting consensus metadata
W20260812 03:16:10.094393 11274 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:16:10.093616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:33917 (local address 127.0.0.1:53760)
0812 03:16:10.093766 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:10.093784 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:10.093889 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:10.093993 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:33917: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 03:16:10.094666 10434 catalog_manager.cc:4584] TS 8c28bf7633604af6af8ae6a2a7d6a3f5 (127.16.217.65:33917): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f4b1325c29b4875843a330e6a1058d5: Network error: Client connection negotiation failed: client connection to 127.16.217.65:33917: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260812 03:16:10.095275 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 10810
I20260812 03:16:10.114492 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 11045
I20260812 03:16:10.117942 11274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:10.114829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:37295 (local address 127.0.0.1:53080)
0812 03:16:10.117716 (+ 2887us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:10.117827 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.66:37295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2755}
I20260812 03:16:10.131516 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 10404
I20260812 03:16:10.146878 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:43829
--webserver_interface=127.16.217.126
--webserver_port=33957
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:43829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:16:10.266660 11323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:10.266952 11323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:10.266994 11323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:10.271196 11323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:16:10.271272 11323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:10.271293 11323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:10.271310 11323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:16:10.271327 11323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:16:10.277794 11323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:43829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:43829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=33957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:10.279071 11323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:10.280145 11323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:10.288861 11337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:10.289176 11323 server_base.cc:1034] running on GCE node
W20260812 03:16:10.288879 11334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:10.288879 11333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:10.289966 11323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:10.291165 11323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:10.292366 11323 hybrid_clock.cc:648] HybridClock initialized: now 1786504570292325 us; error 44 us; skew 500 ppm
Aug 12 03:16:10 dist-test-slave-7nm7 krb5kdc[9399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504570, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:10.297621 11323 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:16:10.298825 11323 webserver.cc:460] Webserver started at http://127.16.217.126:33957/ using document root <none> and password file <none>
I20260812 03:16:10.299449 11323 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:10.299515 11323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:10.303663 11323 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:16:10.306010 11356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:10.307505 11323 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:16:10.307651 11323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "2357121729f14e2ab7c79d86fc24e4e7"
format_stamp: "Formatted at 2026-08-12 03:16:07 on dist-test-slave-7nm7"
server_key: "9254aebe872a6a4221b293c577530f90"
server_key_iv: "d3089256b370854b054f7f2e019e107e"
server_key_version: "encryptionkey@0"
I20260812 03:16:10.308259 11323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:10.345705 11323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:10.349488 11323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:10.349722 11323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:10.358332 11471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:43829 every 8 connection(s)
I20260812 03:16:10.358340 11323 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:43829
I20260812 03:16:10.359606 11323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:16:10.363976 11474 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:16:10.365298 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 11323
I20260812 03:16:10.365760 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:33917
--local_ip_for_outbound_sockets=127.16.217.65
--tserver_master_addrs=127.16.217.126:43829
--webserver_port=45749
--webserver_interface=127.16.217.65
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:16:10.365944 11474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7: Bootstrap starting.
I20260812 03:16:10.385142 11474 log.cc:824] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:10.397859 11474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:16:10.398380 11474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7: Bootstrap complete.
I20260812 03:16:10.402976 11474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2357121729f14e2ab7c79d86fc24e4e7" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43829 } }
I20260812 03:16:10.403375 11474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2357121729f14e2ab7c79d86fc24e4e7, State: Initialized, Role: FOLLOWER
I20260812 03:16:10.403927 11474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2357121729f14e2ab7c79d86fc24e4e7" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43829 } }
I20260812 03:16:10.404047 11474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:16:10.404134 11474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:16:10.404253 11474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 2 FOLLOWER]: Advancing to term 3
I20260812 03:16:10.406383 11474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2357121729f14e2ab7c79d86fc24e4e7" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43829 } }
I20260812 03:16:10.406730 11474 leader_election.cc:302] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2357121729f14e2ab7c79d86fc24e4e7; no voters:
I20260812 03:16:10.407056 11474 leader_election.cc:288] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 03:16:10.407205 11479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 03:16:10.407917 11479 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [term 3 LEADER]: Becoming Leader. State: Replica: 2357121729f14e2ab7c79d86fc24e4e7, State: Running, Role: LEADER
I20260812 03:16:10.408270 11479 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2357121729f14e2ab7c79d86fc24e4e7" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43829 } }
I20260812 03:16:10.408471 11474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:16:10.409762 11480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2357121729f14e2ab7c79d86fc24e4e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2357121729f14e2ab7c79d86fc24e4e7" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43829 } } }
I20260812 03:16:10.409893 11480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [sys.catalog]: This master's current role is: LEADER
I20260812 03:16:10.410995 11516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:16:10.411367 11483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2357121729f14e2ab7c79d86fc24e4e7. Latest consensus state: current_term: 3 leader_uuid: "2357121729f14e2ab7c79d86fc24e4e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2357121729f14e2ab7c79d86fc24e4e7" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43829 } } }
I20260812 03:16:10.411638 11483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2357121729f14e2ab7c79d86fc24e4e7 [sys.catalog]: This master's current role is: LEADER
I20260812 03:16:10.416921 11516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:16:10.417631 11516 catalog_manager.cc:1259] Loaded cluster ID: 61b72f09f1d047dc8ea9ec918dde15a8
I20260812 03:16:10.417698 11516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 03:16:10.418828 11516 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f5d152b99ad google::LogMessage::Fail() at ??:0
@ 0x7f5d152bc88a google::LogMessage::SendToLog() at ??:0
@ 0x7f5d152b94de google::LogMessage::Flush() at ??:0
@ 0x7f5d152baa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5d1fbf8fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f5d1fbf9d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5d1fbf6579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f5d1fc3a640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f5d1750b3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5d1750bc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f5d1750d3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f5d174fc512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5d18b406db start_thread at ??:0
@ 0x7f5d13db761f clone at ??:0
W20260812 03:16:10.517278 11476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:10.517594 11476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:10.517670 11476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:10.521847 11476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:10.521950 11476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:10.522047 11476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:16:10.527846 11476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:33917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=45749
--tserver_master_addrs=127.16.217.126:43829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:10.529035 11476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:10.530439 11476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:10.542735 11571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:10.542850 11564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:10.543123 11476 server_base.cc:1034] running on GCE node
W20260812 03:16:10.542708 11565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:10.543672 11476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:10.544373 11476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:10.545689 11476 hybrid_clock.cc:648] HybridClock initialized: now 1786504570545643 us; error 57 us; skew 500 ppm
Aug 12 03:16:10 dist-test-slave-7nm7 krb5kdc[9399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504570, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:10.549211 11476 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:16:10.550464 11476 webserver.cc:460] Webserver started at http://127.16.217.65:45749/ using document root <none> and password file <none>
I20260812 03:16:10.551087 11476 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:10.551154 11476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:10.555333 11476 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:16:10.557843 11582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:10.559381 11476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:16:10.559538 11476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5"
format_stamp: "Formatted at 2026-08-12 03:16:07 on dist-test-slave-7nm7"
server_key: "12873e39c3717afb2a1fdefeaa2a18a0"
server_key_iv: "11169fb248d24c443ef142d09561d222"
server_key_version: "encryptionkey@0"
I20260812 03:16:10.560118 11476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:10.585594 11476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:10.589228 11476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:10.589476 11476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:10.590332 11476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:16:10.591774 11597 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:16:10.596724 11476 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:16:10.596792 11476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260812 03:16:10.596844 11476 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:16:10.599277 11476 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:16:10.599349 11476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260812 03:16:10.599628 11597 tablet_bootstrap.cc:492] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Bootstrap starting.
I20260812 03:16:10.611712 11476 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:33917
I20260812 03:16:10.611937 11792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:33917 every 8 connection(s)
I20260812 03:16:10.613013 11476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:16:10.617044 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 11476
I20260812 03:16:10.617671 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:37295
--local_ip_for_outbound_sockets=127.16.217.66
--tserver_master_addrs=127.16.217.126:43829
--webserver_port=44161
--webserver_interface=127.16.217.66
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:16:10.624675 11597 log.cc:824] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:10.635540 11597 tablet_bootstrap.cc:492] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:16:10.636005 11597 tablet_bootstrap.cc:492] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Bootstrap complete.
I20260812 03:16:10.637516 11597 ts_tablet_manager.cc:1397] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Time spent bootstrapping tablet: real 0.038s user 0.020s sys 0.013s
I20260812 03:16:10.640838 11597 raft_consensus.cc:348] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } } peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } }
I20260812 03:16:10.641463 11597 raft_consensus.cc:729] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c28bf7633604af6af8ae6a2a7d6a3f5, State: Initialized, Role: FOLLOWER
I20260812 03:16:10.642000 11597 consensus_queue.cc:260] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } } peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } }
I20260812 03:16:10.643114 11597 ts_tablet_manager.cc:1428] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5: Time spent starting tablet: real 0.006s user 0.005s sys 0.003s
W20260812 03:16:10.662170 11798 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:16:10.615542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:40125)
0812 03:16:10.616212 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:10.616274 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:10.617445 (+ 1171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:10.661406 (+ 43961us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":189,"threads_started":1}
W20260812 03:16:10.662854 11796 heartbeater.cc:644] Failed to heartbeat to 127.16.217.126:43829 (0 consecutive failures): Network error: Failed to ping master at 127.16.217.126:43829: Client connection negotiation failed: client connection to 127.16.217.126:43829: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 03:16:10.663606 11798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:10.663340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:57891)
0812 03:16:10.663474 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:10.663551 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 03:16:10.664327 11798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:10.664132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:40533)
0812 03:16:10.664220 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:10.664276 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 03:16:10.664535 11796 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:16:10.750936 11799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:10.751685 11799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:10.752003 11799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:10.756678 11799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:10.756750 11799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:10.756875 11799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:16:10.763923 11799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:37295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=44161
--tserver_master_addrs=127.16.217.126:43829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:10.765234 11799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:10.766376 11799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:10.777760 11815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:10.777769 11814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:10.777814 11819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:10.778322 11799 server_base.cc:1034] running on GCE node
I20260812 03:16:10.778786 11799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:10.779531 11799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:10.780733 11799 hybrid_clock.cc:648] HybridClock initialized: now 1786504570780714 us; error 35 us; skew 500 ppm
Aug 12 03:16:10 dist-test-slave-7nm7 krb5kdc[9399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504570, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:10.787541 11799 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:16:10.788815 11799 webserver.cc:460] Webserver started at http://127.16.217.66:44161/ using document root <none> and password file <none>
I20260812 03:16:10.789458 11799 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:10.789558 11799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:10.794340 11799 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:16:10.796587 11834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:10.798342 11799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:16:10.798488 11799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "291a4e1058664a08acc10f111160dafc"
format_stamp: "Formatted at 2026-08-12 03:16:08 on dist-test-slave-7nm7"
server_key: "fc0aa1ad542f6c7a20e1d7d2bdd99e4b"
server_key_iv: "cc7232f85f50fc6a43efa35b05a5f7fc"
server_key_version: "encryptionkey@0"
I20260812 03:16:10.799077 11799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:10.817801 11799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:10.821403 11799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:10.821657 11799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:10.822485 11799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:16:10.823565 11799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:16:10.823652 11799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:10.823732 11799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:16:10.823779 11799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:10.836879 11799 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:37295
I20260812 03:16:10.836961 12036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:37295 every 8 connection(s)
I20260812 03:16:10.838105 11799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:16:10.838649 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 11799
I20260812 03:16:10.839169 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:43981
--local_ip_for_outbound_sockets=127.16.217.67
--tserver_master_addrs=127.16.217.126:43829
--webserver_port=41767
--webserver_interface=127.16.217.67
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:16:10.842605 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:10.840643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:50589)
0812 03:16:10.841265 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:10.841753 (+ 488us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":125,"threads_started":1}
W20260812 03:16:10.843178 12040 heartbeater.cc:644] Failed to heartbeat to 127.16.217.126:43829 (0 consecutive failures): Network error: Failed to ping master at 127.16.217.126:43829: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
I20260812 03:16:10.843873 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:10.843636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:52071)
0812 03:16:10.843757 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:10.843810 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 03:16:10.844533 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:10.844320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:47045)
0812 03:16:10.844433 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:10.844478 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 03:16:10.844725 12040 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:16:10.964880 12044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:10.965209 12044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:10.965287 12044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:10.969429 12044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:10.969529 12044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:10.969631 12044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:16:10.975214 12044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:43981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=41767
--tserver_master_addrs=127.16.217.126:43829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:10.976388 12044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:10.977447 12044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:10.987416 12056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:10.987411 12057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:10.987411 12060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:10.987653 12044 server_base.cc:1034] running on GCE node
I20260812 03:16:10.988103 12044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:10.988777 12044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:10.990003 12044 hybrid_clock.cc:648] HybridClock initialized: now 1786504570989983 us; error 35 us; skew 500 ppm
Aug 12 03:16:10 dist-test-slave-7nm7 krb5kdc[9399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504570, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:10.993196 12044 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:16:10.994355 12044 webserver.cc:460] Webserver started at http://127.16.217.67:41767/ using document root <none> and password file <none>
I20260812 03:16:10.994990 12044 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:10.995087 12044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:10.999356 12044 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:16:11.002184 12072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:11.003779 12044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:16:11.003930 12044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "ca2e58ef291f48ae89775f5b8f360384"
format_stamp: "Formatted at 2026-08-12 03:16:08 on dist-test-slave-7nm7"
server_key: "dbbcc1c906c9b194fb11a83381f26f8b"
server_key_iv: "2e45eb4082da5bd7022325be892fd7ec"
server_key_version: "encryptionkey@0"
I20260812 03:16:11.004490 12044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:11.029840 12044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:11.033347 12044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:11.033572 12044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:11.034266 12044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:16:11.035181 12044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:16:11.035257 12044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:11.035348 12044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:16:11.035391 12044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:11.044866 12044 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:43981
I20260812 03:16:11.044906 12283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:43981 every 8 connection(s)
I20260812 03:16:11.045899 12044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:16:11.049048 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 12044
I20260812 03:16:11.050124 12289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:11.048205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:42705)
0812 03:16:11.048720 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:11.049163 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":129,"threads_started":1}
W20260812 03:16:11.050724 12286 heartbeater.cc:644] Failed to heartbeat to 127.16.217.126:43829 (0 consecutive failures): Network error: Failed to ping master at 127.16.217.126:43829: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
I20260812 03:16:11.051384 12289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:11.051146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:37319)
0812 03:16:11.051276 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:11.051327 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 03:16:11.052117 12289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:11.051881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:59915)
0812 03:16:11.052011 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:11.052062 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 03:16:11.052454 12286 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 03:16:11.665767 12293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:11.665248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:40503)
0812 03:16:11.665557 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:11.665622 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":86,"threads_started":1}
I20260812 03:16:11.845728 12294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:11.845282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:60073)
0812 03:16:11.845553 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:11.845613 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":71,"threads_started":1}
I20260812 03:16:12.053911 12297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:12.053396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:54831)
0812 03:16:12.053722 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:12.053790 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":79,"threads_started":1}
I20260812 03:16:12.285792 12298 raft_consensus.cc:482] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:16:12.285990 12298 raft_consensus.cc:504] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } } peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } }
I20260812 03:16:12.287125 12298 leader_election.cc:288] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca2e58ef291f48ae89775f5b8f360384 (127.16.217.67:43981), 291a4e1058664a08acc10f111160dafc (127.16.217.66:37295)
Aug 12 03:16:12 dist-test-slave-7nm7 krb5kdc[9399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504570, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.67@KRBTEST.COM
Aug 12 03:16:12 dist-test-slave-7nm7 krb5kdc[9399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504570, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.66@KRBTEST.COM
I20260812 03:16:12.302531 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:12.288037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:60667 (local address 127.16.217.66:37295)
0812 03:16:12.288380 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:12.288402 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:12.288432 (+ 30us) server_negotiation.cc:407] Connection header received
0812 03:16:12.289042 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:12.289048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:12.289504 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:12.289781 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:12.290856 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:12.291811 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:12.293225 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:12.293683 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:12.293850 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:12.296828 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:12.296835 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:12.296848 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:12.296883 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:12.299141 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:12.300494 (+ 1353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:12.300502 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:12.300511 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:12.300665 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:12.301036 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:12.301040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:12.301041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:12.301413 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:12.301518 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:12.302023 (+ 505us) server_negotiation.cc:299] Negotiation successful
0812 03:16:12.302334 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":96,"threads_started":1}
I20260812 03:16:12.302568 12304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:12.287817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:37295 (local address 127.16.217.65:60667)
0812 03:16:12.288109 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:12.288120 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:12.288236 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:12.289957 (+ 1721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:12.289961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:12.290149 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:12.290677 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:12.290686 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:12.291962 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:12.291970 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:12.292896 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:12.292965 (+ 69us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:12.293826 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:12.293832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:12.294030 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:12.294670 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:12.294687 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:12.296624 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:12.299301 (+ 2677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:12.299309 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:12.299324 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:12.299873 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:12.300808 (+ 935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:12.300811 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:12.300812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:12.300917 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:12.301517 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:12.301520 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:12.301703 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 03:16:12.301926 (+ 223us) client_negotiation.cc:240] Negotiation successful
0812 03:16:12.302328 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":149,"spinlock_wait_cycles":46080,"thread_start_us":71,"threads_started":1}
I20260812 03:16:12.302568 12299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:12.287420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:43981 (local address 127.16.217.65:39127)
0812 03:16:12.287668 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:12.287685 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:12.287798 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:12.289580 (+ 1782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:12.289586 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:12.290113 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:12.290667 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:12.290689 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:12.291963 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:12.291969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:12.292898 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:12.292910 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:12.293889 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:12.293893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:12.294047 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:12.294834 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:12.294847 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:12.296334 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:12.299301 (+ 2967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:12.299309 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:12.299324 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:12.299873 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:12.300425 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:12.300431 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:12.300436 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:12.300596 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:12.301367 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:12.301373 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:12.301703 (+ 330us) client_negotiation.cc:769] Sending connection context
0812 03:16:12.301926 (+ 223us) client_negotiation.cc:240] Negotiation successful
0812 03:16:12.302402 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":297,"spinlock_wait_cycles":41984,"thread_start_us":103,"threads_started":1}
I20260812 03:16:12.303467 12300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:12.287559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:39127 (local address 127.16.217.67:43981)
0812 03:16:12.287996 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:12.288026 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:12.288082 (+ 56us) server_negotiation.cc:407] Connection header received
0812 03:16:12.288832 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:12.288838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:12.289184 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:12.289497 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:12.290858 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:12.291810 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:12.293168 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:12.293712 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:12.293924 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:12.296606 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:12.296611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:12.296624 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:12.296654 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:12.299141 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:12.300177 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:12.300183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:12.300189 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:12.300314 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:12.300706 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:12.300710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:12.300713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:12.301243 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:12.301395 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:12.302960 (+ 1565us) server_negotiation.cc:299] Negotiation successful
0812 03:16:12.303276 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":94,"threads_started":1}
W20260812 03:16:12.305801 11583 leader_election.cc:341] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca2e58ef291f48ae89775f5b8f360384 (127.16.217.67:43981): Not found: Tablet not found: 3f4b1325c29b4875843a330e6a1058d5
W20260812 03:16:12.305989 11583 leader_election.cc:341] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 291a4e1058664a08acc10f111160dafc (127.16.217.66:37295): Not found: Tablet not found: 3f4b1325c29b4875843a330e6a1058d5
I20260812 03:16:12.306035 11583 leader_election.cc:302] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c28bf7633604af6af8ae6a2a7d6a3f5; no voters: 291a4e1058664a08acc10f111160dafc, ca2e58ef291f48ae89775f5b8f360384
I20260812 03:16:12.306326 12298 raft_consensus.cc:2726] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:16:12.667259 12304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:12.666768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:38223)
0812 03:16:12.666910 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:12.666986 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 03:16:12.847240 12310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:12.846694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:49235)
0812 03:16:12.846982 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:12.847097 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":91,"threads_started":1}
I20260812 03:16:13.055380 12311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:13.054911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:47421)
0812 03:16:13.055216 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:13.055284 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":95,"threads_started":1}
I20260812 03:16:13.668721 12313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:13.668257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:44201)
0812 03:16:13.668539 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:13.668602 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":83,"threads_started":1}
I20260812 03:16:13.848448 12314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:13.847975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:37417)
0812 03:16:13.848264 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:13.848327 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":77,"threads_started":1}
I20260812 03:16:14.056619 12315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:14.056175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:33205)
0812 03:16:14.056449 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:14.056505 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":80,"threads_started":1}
I20260812 03:16:14.480535 12318 raft_consensus.cc:482] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:16:14.480693 12318 raft_consensus.cc:504] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } } peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } }
I20260812 03:16:14.481098 12318 leader_election.cc:288] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca2e58ef291f48ae89775f5b8f360384 (127.16.217.67:43981), 291a4e1058664a08acc10f111160dafc (127.16.217.66:37295)
W20260812 03:16:14.482111 11583 leader_election.cc:341] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca2e58ef291f48ae89775f5b8f360384 (127.16.217.67:43981): Not found: Tablet not found: 3f4b1325c29b4875843a330e6a1058d5
W20260812 03:16:14.482511 11583 leader_election.cc:341] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 291a4e1058664a08acc10f111160dafc (127.16.217.66:37295): Not found: Tablet not found: 3f4b1325c29b4875843a330e6a1058d5
I20260812 03:16:14.482662 11583 leader_election.cc:302] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c28bf7633604af6af8ae6a2a7d6a3f5; no voters: 291a4e1058664a08acc10f111160dafc, ca2e58ef291f48ae89775f5b8f360384
I20260812 03:16:14.483013 12318 raft_consensus.cc:2726] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:16:14.670593 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:14.669977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:35931)
0812 03:16:14.670399 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:14.670464 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":140,"threads_started":1}
I20260812 03:16:14.849879 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:14.849343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:36691)
0812 03:16:14.849673 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:14.849754 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":95,"threads_started":1}
I20260812 03:16:15.058175 12322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:15.057637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:59677)
0812 03:16:15.058001 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:15.058068 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":105,"threads_started":1}
I20260812 03:16:15.672003 12325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:15.671519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:53643)
0812 03:16:15.671819 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:15.671885 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":75,"threads_started":1}
I20260812 03:16:15.851282 12326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:15.850805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:47177)
0812 03:16:15.851100 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:15.851165 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":85,"threads_started":1}
I20260812 03:16:16.059543 12327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:16.059050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:36777)
0812 03:16:16.059372 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:16.059444 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":95,"threads_started":1}
I20260812 03:16:16.673424 12328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:16.672944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:55363)
0812 03:16:16.673271 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:16.673327 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":140,"threads_started":1}
I20260812 03:16:16.852638 12329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:16.852182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:47255)
0812 03:16:16.852468 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:16.852532 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":96,"threads_started":1}
I20260812 03:16:17.060756 12330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:17.060359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:41395)
0812 03:16:17.060606 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:17.060661 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":77,"threads_started":1}
I20260812 03:16:17.249634 12332 raft_consensus.cc:482] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:16:17.249773 12332 raft_consensus.cc:504] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } } peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } }
I20260812 03:16:17.250147 12332 leader_election.cc:288] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca2e58ef291f48ae89775f5b8f360384 (127.16.217.67:43981), 291a4e1058664a08acc10f111160dafc (127.16.217.66:37295)
W20260812 03:16:17.251152 11583 leader_election.cc:341] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca2e58ef291f48ae89775f5b8f360384 (127.16.217.67:43981): Not found: Tablet not found: 3f4b1325c29b4875843a330e6a1058d5
W20260812 03:16:17.251312 11583 leader_election.cc:341] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 291a4e1058664a08acc10f111160dafc (127.16.217.66:37295): Not found: Tablet not found: 3f4b1325c29b4875843a330e6a1058d5
I20260812 03:16:17.251377 11583 leader_election.cc:302] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c28bf7633604af6af8ae6a2a7d6a3f5; no voters: 291a4e1058664a08acc10f111160dafc, ca2e58ef291f48ae89775f5b8f360384
I20260812 03:16:17.251577 12332 raft_consensus.cc:2726] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:16:17.674780 12333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:17.674343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:45253)
0812 03:16:17.674614 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:17.674677 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":75,"threads_started":1}
I20260812 03:16:17.854149 12336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:17.853609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:45269)
0812 03:16:17.853947 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:17.854043 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":80,"threads_started":1}
I20260812 03:16:18.062136 12338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:18.061602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:45839)
0812 03:16:18.061946 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:18.062010 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":83,"threads_started":1}
I20260812 03:16:18.676090 12340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:18.675620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:51989)
0812 03:16:18.675918 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:18.675990 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":86,"threads_started":1}
I20260812 03:16:18.830984 12342 raft_consensus.cc:482] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:16:18.831146 12342 raft_consensus.cc:504] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } } peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } }
I20260812 03:16:18.831606 12342 leader_election.cc:288] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca2e58ef291f48ae89775f5b8f360384 (127.16.217.67:43981), 291a4e1058664a08acc10f111160dafc (127.16.217.66:37295)
W20260812 03:16:18.832432 11583 leader_election.cc:341] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 291a4e1058664a08acc10f111160dafc (127.16.217.66:37295): Not found: Tablet not found: 3f4b1325c29b4875843a330e6a1058d5
W20260812 03:16:18.832531 11583 leader_election.cc:341] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca2e58ef291f48ae89775f5b8f360384 (127.16.217.67:43981): Not found: Tablet not found: 3f4b1325c29b4875843a330e6a1058d5
I20260812 03:16:18.832567 11583 leader_election.cc:302] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c28bf7633604af6af8ae6a2a7d6a3f5; no voters: 291a4e1058664a08acc10f111160dafc, ca2e58ef291f48ae89775f5b8f360384
I20260812 03:16:18.832715 12342 raft_consensus.cc:2726] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:16:18.855535 12343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:18.855098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:35085)
0812 03:16:18.855375 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:18.855435 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":80,"threads_started":1}
I20260812 03:16:19.063498 12344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:19.062982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:44279)
0812 03:16:19.063263 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:19.063326 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":82,"threads_started":1}
I20260812 03:16:19.677578 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:19.677096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:47711)
0812 03:16:19.677414 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:19.677473 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":108,"threads_started":1}
I20260812 03:16:19.857023 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:19.856560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:34401)
0812 03:16:19.856845 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:19.856919 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":86,"threads_started":1}
I20260812 03:16:20.065006 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:20.064505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:54019)
0812 03:16:20.064838 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:20.064906 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":102,"threads_started":1}
I20260812 03:16:20.678877 12350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:20.678497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:47125)
0812 03:16:20.678730 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:20.678784 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":73,"threads_started":1}
I20260812 03:16:20.858481 12352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:20.857972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:38609)
0812 03:16:20.858314 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:20.858378 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":91,"threads_started":1}
I20260812 03:16:21.066565 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:21.065839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:51931)
0812 03:16:21.066412 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:21.066468 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":108,"threads_started":1}
I20260812 03:16:21.160486 12361 raft_consensus.cc:482] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:16:21.160621 12361 raft_consensus.cc:504] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2e58ef291f48ae89775f5b8f360384" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 43981 } } peers { permanent_uuid: "291a4e1058664a08acc10f111160dafc" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 37295 } } peers { permanent_uuid: "8c28bf7633604af6af8ae6a2a7d6a3f5" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 33917 } }
I20260812 03:16:21.161001 12361 leader_election.cc:288] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca2e58ef291f48ae89775f5b8f360384 (127.16.217.67:43981), 291a4e1058664a08acc10f111160dafc (127.16.217.66:37295)
W20260812 03:16:21.161991 11583 leader_election.cc:341] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca2e58ef291f48ae89775f5b8f360384 (127.16.217.67:43981): Not found: Tablet not found: 3f4b1325c29b4875843a330e6a1058d5
W20260812 03:16:21.162195 11583 leader_election.cc:341] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 291a4e1058664a08acc10f111160dafc (127.16.217.66:37295): Not found: Tablet not found: 3f4b1325c29b4875843a330e6a1058d5
I20260812 03:16:21.162295 11583 leader_election.cc:302] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c28bf7633604af6af8ae6a2a7d6a3f5; no voters: 291a4e1058664a08acc10f111160dafc, ca2e58ef291f48ae89775f5b8f360384
I20260812 03:16:21.162493 12361 raft_consensus.cc:2726] T 3f4b1325c29b4875843a330e6a1058d5 P 8c28bf7633604af6af8ae6a2a7d6a3f5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:16:21.680387 12364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:21.679811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:48373)
0812 03:16:21.680198 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:21.680272 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":85,"threads_started":1}
I20260812 03:16:21.859941 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:21.859476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:38313)
0812 03:16:21.859781 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:21.859843 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":126,"threads_started":1}
I20260812 03:16:22.067947 12368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:22.067462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:60739)
0812 03:16:22.067775 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:22.067843 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":100,"threads_started":1}
I20260812 03:16:22.681735 12370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:22.681300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:45077)
0812 03:16:22.681570 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:22.681631 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":74,"threads_started":1}
I20260812 03:16:22.861249 12371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:22.860825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:38607)
0812 03:16:22.861087 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:22.861151 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":83,"threads_started":1}
I20260812 03:16:23.069278 12372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:23.068794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:37787)
0812 03:16:23.069104 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:23.069170 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":96,"threads_started":1}
I20260812 03:16:23.683280 12373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:23.682794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:53909)
0812 03:16:23.683095 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:23.683172 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":79,"threads_started":1}
I20260812 03:16:23.862782 12376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:23.862200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:45459)
0812 03:16:23.862601 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:23.862677 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":102,"threads_started":1}
I20260812 03:16:24.070802 12378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:24.070342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:38005)
0812 03:16:24.070633 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:24.070702 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":99,"threads_started":1}
I20260812 03:16:24.684696 12379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:24.684238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:49459)
0812 03:16:24.684529 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:24.684592 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":87,"threads_started":1}
I20260812 03:16:24.864032 12380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:24.863613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:39053)
0812 03:16:24.863860 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:24.863915 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":81,"threads_started":1}
I20260812 03:16:25.072348 12381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:25.071771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:44705)
0812 03:16:25.072082 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:25.072162 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":85,"threads_started":1}
I20260812 03:16:25.686216 12382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:25.685775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.65:54705)
0812 03:16:25.686033 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:25.686095 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":76,"threads_started":1}
I20260812 03:16:25.865814 12384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:25.865139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.66:57289)
0812 03:16:25.865632 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:25.865705 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":111,"threads_started":1}
I20260812 03:16:26.060702 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 11476
I20260812 03:16:26.077076 12385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:26.073741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:43829 (local address 127.16.217.67:56877)
0812 03:16:26.076851 (+ 3110us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:26.076967 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2997,"thread_start_us":1272,"threads_started":1}
I20260812 03:16:26.083900 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 11799
I20260812 03:16:26.102193 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 12044
2026-08-12T03:16:26Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (20254 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:16:26 dist-test-slave-7nm7 krb5kdc[12390](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:16:26 dist-test-slave-7nm7 krb5kdc[12390](info): set up 2 sockets
Aug 12 03:16:26 dist-test-slave-7nm7 krb5kdc[12390](info): commencing operation
krb5kdc: starting...
W20260812 03:16:27.464465 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:16:27 dist-test-slave-7nm7 krb5kdc[12390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:16:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:16:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:16:27.609473 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:39545
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:46323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:39545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:16:27.734273 12410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:27.734558 12410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:27.734599 12410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:27.738745 12410 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 03:16:27.738799 12410 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 03:16:27.738816 12410 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 03:16:27.738829 12410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:16:27.738847 12410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:27.738865 12410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:27.738880 12410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:16:27.738895 12410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:16:27.744164 12410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:46323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:39545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:39545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:27.745283 12410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:27.746307 12410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:27.755509 12419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:27.755581 12418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:27.755515 12410 server_base.cc:1034] running on GCE node
W20260812 03:16:27.755575 12422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:27.756588 12410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:27.757774 12410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:27.758958 12410 hybrid_clock.cc:648] HybridClock initialized: now 1786504587758930 us; error 44 us; skew 500 ppm
Aug 12 03:16:27 dist-test-slave-7nm7 krb5kdc[12390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504587, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:27.762045 12410 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:16:27.763163 12410 webserver.cc:460] Webserver started at http://127.16.217.126:42413/ using document root <none> and password file <none>
I20260812 03:16:27.763762 12410 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:27.763830 12410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:27.764068 12410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:16:27.766286 12410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "1a246a48a5fd44a78d33e137849e7707"
format_stamp: "Formatted at 2026-08-12 03:16:27 on dist-test-slave-7nm7"
server_key: "c93a6b2b3d7172146120e4503cfb80b1"
server_key_iv: "e91a8d041fcddbe8cfdaa19803619f1d"
server_key_version: "encryptionkey@0"
I20260812 03:16:27.766930 12410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "1a246a48a5fd44a78d33e137849e7707"
format_stamp: "Formatted at 2026-08-12 03:16:27 on dist-test-slave-7nm7"
server_key: "c93a6b2b3d7172146120e4503cfb80b1"
server_key_iv: "e91a8d041fcddbe8cfdaa19803619f1d"
server_key_version: "encryptionkey@0"
I20260812 03:16:27.771236 12410 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 03:16:27.773787 12434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:27.775185 12410 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:16:27.775331 12410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "1a246a48a5fd44a78d33e137849e7707"
format_stamp: "Formatted at 2026-08-12 03:16:27 on dist-test-slave-7nm7"
server_key: "c93a6b2b3d7172146120e4503cfb80b1"
server_key_iv: "e91a8d041fcddbe8cfdaa19803619f1d"
server_key_version: "encryptionkey@0"
I20260812 03:16:27.775453 12410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:27.805912 12410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:27.809517 12410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:27.809772 12410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:27.817431 12410 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:39545
I20260812 03:16:27.817790 12531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:39545 every 8 connection(s)
I20260812 03:16:27.818521 12410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:16:27.818953 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 12410
I20260812 03:16:27.819130 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:16:27.819479 17253 external_mini_cluster.cc:1428] Setting key e3104101175b583e4b0ace7a16d1aa9b
I20260812 03:16:27.821986 12533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:16:27.828887 12533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: Bootstrap starting.
Aug 12 03:16:27 dist-test-slave-7nm7 krb5kdc[12390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:16:27.831740 12533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: Neither blocks nor log segments found. Creating new log.
I20260812 03:16:27.832545 12533 log.cc:824] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:27.834942 12533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: No bootstrap required, opened a new log
I20260812 03:16:27.837289 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:27.820961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45208 (local address 127.16.217.126:39545)
0812 03:16:27.821387 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:27.821413 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:27.821456 (+ 43us) server_negotiation.cc:407] Connection header received
0812 03:16:27.822190 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:27.822209 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:27.822609 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:27.822890 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:27.823922 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:27.825752 (+ 1830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:27.827112 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:27.828090 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:27.828265 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:27.831598 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:27.831604 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:27.831620 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:27.831652 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:27.833717 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:27.834213 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:27.834219 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:27.834225 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:27.834397 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:27.834714 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:27.834717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:27.834719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:27.835245 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:27.835455 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:27.835771 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 03:16:27.836235 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":140,"threads_started":1}
I20260812 03:16:27.839515 12533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a246a48a5fd44a78d33e137849e7707" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39545 } }
I20260812 03:16:27.839816 12533 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:16:27.839921 12533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a246a48a5fd44a78d33e137849e7707, State: Initialized, Role: FOLLOWER
I20260812 03:16:27.840384 12533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a246a48a5fd44a78d33e137849e7707" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39545 } }
I20260812 03:16:27.840517 12533 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:16:27.840561 12533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:16:27.840677 12533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:16:27.841881 12533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a246a48a5fd44a78d33e137849e7707" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39545 } }
I20260812 03:16:27.842321 12533 leader_election.cc:302] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a246a48a5fd44a78d33e137849e7707; no voters:
I20260812 03:16:27.842671 12533 leader_election.cc:288] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:16:27.842903 12540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:16:27.843231 12540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 1 LEADER]: Becoming Leader. State: Replica: 1a246a48a5fd44a78d33e137849e7707, State: Running, Role: LEADER
I20260812 03:16:27.843547 12540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a246a48a5fd44a78d33e137849e7707" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39545 } }
I20260812 03:16:27.844054 12533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:16:27.845335 12542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a246a48a5fd44a78d33e137849e7707. Latest consensus state: current_term: 1 leader_uuid: "1a246a48a5fd44a78d33e137849e7707" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a246a48a5fd44a78d33e137849e7707" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39545 } } }
I20260812 03:16:27.845471 12542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [sys.catalog]: This master's current role is: LEADER
I20260812 03:16:27.846139 12541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a246a48a5fd44a78d33e137849e7707" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a246a48a5fd44a78d33e137849e7707" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39545 } } }
I20260812 03:16:27.846264 12541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [sys.catalog]: This master's current role is: LEADER
W20260812 03:16:27.848910 12562 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:16:27.849058 12562 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:16:27.849139 12555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:16:27.850010 12555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:16:27.857013 12555 catalog_manager.cc:1347] Generated new cluster ID: 2bea95d08bf047dd91d9c6768a3fe5a5
I20260812 03:16:27.857218 12555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:16:27.879959 12555 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:16:27.881789 12555 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:16:27.892268 12555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: Generated new TSK 0
I20260812 03:16:27.893177 12555 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
I20260812 03:16:27.934269 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39545
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:46323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:16:28.057590 12572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:28.057937 12572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:28.058017 12572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:28.062180 12572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:28.062304 12572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:28.062419 12572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:16:28.068140 12572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:46323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:28.069310 12572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:28.070405 12572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:28.080931 12584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:28.080960 12588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:28.080951 12582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:28.081260 12572 server_base.cc:1034] running on GCE node
I20260812 03:16:28.082003 12572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:28.082752 12572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:28.083928 12572 hybrid_clock.cc:648] HybridClock initialized: now 1786504588083907 us; error 34 us; skew 500 ppm
Aug 12 03:16:28 dist-test-slave-7nm7 krb5kdc[12390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504588, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:28.086969 12572 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:16:28.088032 12572 webserver.cc:460] Webserver started at http://127.16.217.65:34249/ using document root <none> and password file <none>
I20260812 03:16:28.088629 12572 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:28.088728 12572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:28.088973 12572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:16:28.091161 12572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "e4f0fa8043f941ae81e2e2ba831c4814"
format_stamp: "Formatted at 2026-08-12 03:16:28 on dist-test-slave-7nm7"
server_key: "e4fb73f7d157785bbc3be0f79ff92856"
server_key_iv: "9ecf0a21317b38d905140a108ce7236c"
server_key_version: "encryptionkey@0"
I20260812 03:16:28.091742 12572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "e4f0fa8043f941ae81e2e2ba831c4814"
format_stamp: "Formatted at 2026-08-12 03:16:28 on dist-test-slave-7nm7"
server_key: "e4fb73f7d157785bbc3be0f79ff92856"
server_key_iv: "9ecf0a21317b38d905140a108ce7236c"
server_key_version: "encryptionkey@0"
I20260812 03:16:28.095564 12572 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:16:28.097983 12603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:28.099536 12572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:16:28.099691 12572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "e4f0fa8043f941ae81e2e2ba831c4814"
format_stamp: "Formatted at 2026-08-12 03:16:28 on dist-test-slave-7nm7"
server_key: "e4fb73f7d157785bbc3be0f79ff92856"
server_key_iv: "9ecf0a21317b38d905140a108ce7236c"
server_key_version: "encryptionkey@0"
I20260812 03:16:28.099817 12572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:28.116393 12572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:28.120458 12572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:28.120697 12572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:28.121345 12572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:16:28.122306 12572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:16:28.122378 12572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:28.122432 12572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:16:28.122448 12572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:28.132253 12572 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:45103
I20260812 03:16:28.132325 12804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:45103 every 8 connection(s)
I20260812 03:16:28.133245 12572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:16:28.142846 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 12572
I20260812 03:16:28.143002 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
Aug 12 03:16:28 dist-test-slave-7nm7 krb5kdc[12390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504588, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:16:28.143376 17253 external_mini_cluster.cc:1428] Setting key ced159ddfb7d52719611caddb5d3027c
I20260812 03:16:28.148766 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.135221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:59419 (local address 127.16.217.126:39545)
0812 03:16:28.135398 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:28.135401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:28.137097 (+ 1696us) server_negotiation.cc:407] Connection header received
0812 03:16:28.137967 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:28.137970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:28.138027 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:28.138141 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:28.139328 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.139876 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.141026 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.141258 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.141376 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:28.144178 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:28.144181 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:28.144185 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:28.144220 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:28.146005 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:28.146840 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:28.146843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:28.146845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:28.146936 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:28.147428 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:28.147434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:28.147437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:28.147690 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:28.147760 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:28.148356 (+ 596us) server_negotiation.cc:299] Negotiation successful
0812 03:16:28.148549 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 03:16:28.149653 12809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.135944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:59419)
0812 03:16:28.136833 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:28.136876 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:28.137759 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:28.138333 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:28.138340 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:28.138700 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:28.139172 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:28.139185 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.140008 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.140013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:28.140890 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:28.140900 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.141425 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.141430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:28.141602 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:28.142255 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:28.142272 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:28.143995 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:28.146270 (+ 2275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:28.146280 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:28.146301 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:28.146717 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:28.147050 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:28.147056 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:28.147061 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:28.147298 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:28.147821 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:28.147826 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:28.148076 (+ 250us) client_negotiation.cc:769] Sending connection context
0812 03:16:28.148234 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 03:16:28.148566 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":247,"threads_started":1}
I20260812 03:16:28.151117 12806 heartbeater.cc:347] Connected to a master server at 127.16.217.126:39545
I20260812 03:16:28.151386 12806 heartbeater.cc:464] Registering TS with master...
I20260812 03:16:28.151983 12806 heartbeater.cc:511] Master 127.16.217.126:39545 requested a full tablet report, sending...
I20260812 03:16:28.153702 12465 ts_manager.cc:194] Registered new tserver with Master: e4f0fa8043f941ae81e2e2ba831c4814 (127.16.217.65:45103)
WARNING: no policy specified for kudu/127.16.217.66@KRBTEST.COM; defaulting to no policy
I20260812 03:16:28.154964 12465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:59419
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
I20260812 03:16:28.174996 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39545
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:46323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:16:28.299360 12819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:28.299696 12819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:28.299770 12819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:28.304322 12819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:28.304396 12819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:28.304459 12819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:16:28.310428 12819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:46323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:28.311586 12819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:28.312767 12819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:28.323621 12834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:28.323590 12831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:28.323698 12828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:28.323975 12819 server_base.cc:1034] running on GCE node
I20260812 03:16:28.324520 12819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:28.325250 12819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:28.326450 12819 hybrid_clock.cc:648] HybridClock initialized: now 1786504588326423 us; error 53 us; skew 500 ppm
Aug 12 03:16:28 dist-test-slave-7nm7 krb5kdc[12390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504588, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:28.329357 12819 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:16:28.330363 12819 webserver.cc:460] Webserver started at http://127.16.217.66:36211/ using document root <none> and password file <none>
I20260812 03:16:28.330948 12819 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:28.331048 12819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:28.331300 12819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:16:28.333447 12819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "54df08210dcd4b84973990752dc3f55f"
format_stamp: "Formatted at 2026-08-12 03:16:28 on dist-test-slave-7nm7"
server_key: "6ee30a311a7653e64938c1bff06670b0"
server_key_iv: "c26812957020c25a2c38969ba31c4b7d"
server_key_version: "encryptionkey@0"
I20260812 03:16:28.334115 12819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "54df08210dcd4b84973990752dc3f55f"
format_stamp: "Formatted at 2026-08-12 03:16:28 on dist-test-slave-7nm7"
server_key: "6ee30a311a7653e64938c1bff06670b0"
server_key_iv: "c26812957020c25a2c38969ba31c4b7d"
server_key_version: "encryptionkey@0"
I20260812 03:16:28.338323 12819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 03:16:28.341677 12850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:28.343079 12819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 03:16:28.343235 12819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "54df08210dcd4b84973990752dc3f55f"
format_stamp: "Formatted at 2026-08-12 03:16:28 on dist-test-slave-7nm7"
server_key: "6ee30a311a7653e64938c1bff06670b0"
server_key_iv: "c26812957020c25a2c38969ba31c4b7d"
server_key_version: "encryptionkey@0"
I20260812 03:16:28.343358 12819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:28.367826 12819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:28.371376 12819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:28.371618 12819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:28.372287 12819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:16:28.373184 12819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:16:28.373261 12819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:28.373337 12819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:16:28.373382 12819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:28.383195 12819 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:42703
I20260812 03:16:28.383277 13067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:42703 every 8 connection(s)
I20260812 03:16:28.384205 12819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:16:28.393857 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 12819
I20260812 03:16:28.394047 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:16:28.394630 17253 external_mini_cluster.cc:1428] Setting key 44c9201b305c79cc6312eb95da4c5a9a
Aug 12 03:16:28 dist-test-slave-7nm7 krb5kdc[12390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504588, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:16:28.400230 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.386292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:51209 (local address 127.16.217.126:39545)
0812 03:16:28.386427 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:28.386431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:28.387377 (+ 946us) server_negotiation.cc:407] Connection header received
0812 03:16:28.388766 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:28.388769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:28.388848 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:28.388950 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:28.390154 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.390742 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.391921 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.392165 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.392285 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:28.395892 (+ 3607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:28.395895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:28.395898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:28.395922 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:28.397512 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:28.398201 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:28.398204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:28.398205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:28.398305 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:28.398748 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:28.398754 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:28.398758 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:28.399094 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:28.399228 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:28.399833 (+ 605us) server_negotiation.cc:299] Negotiation successful
0812 03:16:28.399983 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:16:28.401161 13076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.386716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:51209)
0812 03:16:28.387205 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:28.387251 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:28.388540 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:28.389136 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:28.389144 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:28.389517 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:28.389998 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:28.390014 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.390879 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.390888 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:28.391784 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:28.391794 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.392332 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.392336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:28.392532 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:28.393348 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:28.393376 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:28.395714 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:28.397641 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:28.397652 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:28.397673 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:28.398088 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:28.398392 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:28.398398 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:28.398403 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:28.398621 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:28.399259 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:28.399291 (+ 32us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:28.399562 (+ 271us) client_negotiation.cc:769] Sending connection context
0812 03:16:28.399756 (+ 194us) client_negotiation.cc:240] Negotiation successful
0812 03:16:28.400195 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":119,"threads_started":1}
I20260812 03:16:28.402598 13071 heartbeater.cc:347] Connected to a master server at 127.16.217.126:39545
I20260812 03:16:28.402894 13071 heartbeater.cc:464] Registering TS with master...
I20260812 03:16:28.403607 13071 heartbeater.cc:511] Master 127.16.217.126:39545 requested a full tablet report, sending...
I20260812 03:16:28.404955 12465 ts_manager.cc:194] Registered new tserver with Master: 54df08210dcd4b84973990752dc3f55f (127.16.217.66:42703)
I20260812 03:16:28.405570 12465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:51209
WARNING: no policy specified for kudu/127.16.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
I20260812 03:16:28.426185 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39545
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:46323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:16:28.567674 13083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:28.568071 13083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:28.568219 13083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:28.573138 13083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:28.573276 13083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:28.573385 13083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:16:28.580157 13083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:46323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:28.581825 13083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:28.582993 13083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:28.593647 13094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:28.593647 13099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:28.593647 13093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:28.594436 13083 server_base.cc:1034] running on GCE node
I20260812 03:16:28.594883 13083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:28.595626 13083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:28.596812 13083 hybrid_clock.cc:648] HybridClock initialized: now 1786504588596788 us; error 35 us; skew 500 ppm
Aug 12 03:16:28 dist-test-slave-7nm7 krb5kdc[12390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504588, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:28.599901 13083 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:16:28.600970 13083 webserver.cc:460] Webserver started at http://127.16.217.67:42005/ using document root <none> and password file <none>
I20260812 03:16:28.601570 13083 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:28.601675 13083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:28.601914 13083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:16:28.604238 13083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "0fd81512c7924e789ad904a4e8c4eb29"
format_stamp: "Formatted at 2026-08-12 03:16:28 on dist-test-slave-7nm7"
server_key: "0971184d74140994b2845cca676b5892"
server_key_iv: "de86c96998fab53b52b02f6c72c41b44"
server_key_version: "encryptionkey@0"
I20260812 03:16:28.604867 13083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "0fd81512c7924e789ad904a4e8c4eb29"
format_stamp: "Formatted at 2026-08-12 03:16:28 on dist-test-slave-7nm7"
server_key: "0971184d74140994b2845cca676b5892"
server_key_iv: "de86c96998fab53b52b02f6c72c41b44"
server_key_version: "encryptionkey@0"
I20260812 03:16:28.609025 13083 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 03:16:28.612011 13110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:28.613438 13083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:16:28.613591 13083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "0fd81512c7924e789ad904a4e8c4eb29"
format_stamp: "Formatted at 2026-08-12 03:16:28 on dist-test-slave-7nm7"
server_key: "0971184d74140994b2845cca676b5892"
server_key_iv: "de86c96998fab53b52b02f6c72c41b44"
server_key_version: "encryptionkey@0"
I20260812 03:16:28.613721 13083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:28.633006 13083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:28.637419 13083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:28.637672 13083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:28.638365 13083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:16:28.639317 13083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:16:28.639389 13083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:28.639462 13083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:16:28.639505 13083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:28.650626 13083 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:40165
I20260812 03:16:28.650653 13318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:40165 every 8 connection(s)
I20260812 03:16:28.651577 13083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:16:28.655472 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 13083
I20260812 03:16:28.655658 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:16:28.655995 17253 external_mini_cluster.cc:1428] Setting key 235b32675e3e23be98ae76e04d4172b8
Aug 12 03:16:28 dist-test-slave-7nm7 krb5kdc[12390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504588, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:16:28.667193 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.653758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:54687 (local address 127.16.217.126:39545)
0812 03:16:28.653924 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:28.653929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:28.655301 (+ 1372us) server_negotiation.cc:407] Connection header received
0812 03:16:28.656783 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:28.656786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:28.656842 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:28.656953 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:28.658605 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.659158 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.660343 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.660556 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.660678 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:28.663292 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:28.663298 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:28.663305 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:28.663331 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:28.664751 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:28.665398 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:28.665402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:28.665405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:28.665507 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:28.665917 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:28.665920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:28.665922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:28.666133 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:28.666286 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:28.666844 (+ 558us) server_negotiation.cc:299] Negotiation successful
0812 03:16:28.666988 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:16:28.668035 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.654446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:54687)
0812 03:16:28.654938 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:28.654984 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:28.656547 (+ 1563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:28.657294 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:28.657302 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:28.657679 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:28.658455 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:28.658474 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.659291 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.659294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:28.660200 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:28.660209 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.660722 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.660726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:28.660895 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:28.661537 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:28.661554 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:28.663116 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:28.664914 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:28.664921 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:28.664937 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:28.665268 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:28.665629 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:28.665633 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:28.665635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:28.665791 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:28.666287 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:28.666296 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:28.666570 (+ 274us) client_negotiation.cc:769] Sending connection context
0812 03:16:28.666784 (+ 214us) client_negotiation.cc:240] Negotiation successful
0812 03:16:28.667154 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":132,"threads_started":1}
I20260812 03:16:28.669538 13319 heartbeater.cc:347] Connected to a master server at 127.16.217.126:39545
I20260812 03:16:28.669770 13319 heartbeater.cc:464] Registering TS with master...
I20260812 03:16:28.670401 13319 heartbeater.cc:511] Master 127.16.217.126:39545 requested a full tablet report, sending...
I20260812 03:16:28.671586 12462 ts_manager.cc:194] Registered new tserver with Master: 0fd81512c7924e789ad904a4e8c4eb29 (127.16.217.67:40165)
I20260812 03:16:28.672226 12462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:54687
I20260812 03:16:28.682504 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:16:28.693977 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.684771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45212 (local address 127.16.217.126:39545)
0812 03:16:28.684918 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:28.684922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:28.685010 (+ 88us) server_negotiation.cc:407] Connection header received
0812 03:16:28.685215 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:28.685218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:28.685280 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:28.685371 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:28.686319 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.686833 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.687895 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.688329 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.688437 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:28.690401 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:28.690405 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:28.690407 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:28.690433 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:28.691911 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:28.692468 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:28.692472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:28.692473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:28.692555 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:28.692852 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:28.692855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:28.692856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:28.693174 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:28.693297 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:28.693615 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 03:16:28.693765 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:16:28.698064 12462 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:16:28.701548 12462 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:16:28.717389 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.710751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:40165 (local address 127.0.0.1:38538)
0812 03:16:28.711147 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:28.711200 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:28.711349 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:28.712543 (+ 1194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:28.712551 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:28.712578 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:28.712729 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:28.712737 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.714145 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.714149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:28.715462 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:28.715475 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.716796 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.716800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:28.716912 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:28.716955 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 03:16:28.717060 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 03:16:28.717191 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":27,"thread_start_us":87,"threads_started":1}
I20260812 03:16:28.717734 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.710965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:42703 (local address 127.0.0.1:39356)
0812 03:16:28.711505 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:28.711521 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:28.711610 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:28.713156 (+ 1546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:28.713159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:28.713180 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:28.713271 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:28.713277 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.714456 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.714460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:28.715421 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:28.715431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.717149 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.717154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:28.717293 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:28.717324 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 03:16:28.717407 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 03:16:28.717598 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":6,"thread_start_us":138,"threads_started":1}
I20260812 03:16:28.717866 13347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.711902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39356 (local address 127.16.217.66:42703)
0812 03:16:28.712708 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:28.712726 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:28.712751 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:16:28.712825 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:28.712829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:28.713004 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:28.713162 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:28.713393 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.714295 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.715558 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.716784 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.717134 (+ 350us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:28.717374 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:28.717558 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 03:16:28.717706 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":104,"threads_started":1}
I20260812 03:16:28.718588 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.711595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:45103 (local address 127.0.0.1:52608)
0812 03:16:28.711904 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:28.711918 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:28.712000 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:28.713324 (+ 1324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:28.713326 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:28.713371 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:28.713484 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:28.713491 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.714879 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.714883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:28.716021 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:28.716032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.718040 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.718044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:28.718197 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:28.718257 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 03:16:28.718342 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 03:16:28.718438 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":18,"thread_start_us":98,"threads_started":1}
I20260812 03:16:28.718839 13352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.712247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52608 (local address 127.16.217.65:45103)
0812 03:16:28.712727 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:28.712733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:28.712752 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:16:28.712825 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:28.712829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:28.713003 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:28.713154 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:28.713636 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.714751 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.716184 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.716935 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.717084 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:28.717226 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:28.718499 (+ 1273us) server_negotiation.cc:299] Negotiation successful
0812 03:16:28.718615 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":104,"threads_started":1}
I20260812 03:16:28.718998 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.710903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38538 (local address 127.16.217.67:40165)
0812 03:16:28.711861 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:28.711875 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:28.711900 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:16:28.711981 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:28.711987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:28.712224 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:28.712385 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:28.712867 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.713989 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.715610 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.716448 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.716526 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:28.716688 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:28.717628 (+ 940us) server_negotiation.cc:299] Negotiation successful
0812 03:16:28.717855 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":853,"thread_start_us":102,"threads_started":1}
I20260812 03:16:28.721639 13204 tablet_service.cc:1467] Processing CreateTablet for tablet 779c932dfd60445194ee4b0e45879f23 (DEFAULT_TABLE table=test-table [id=c3d84190b3f4474ba3cc886a672dee2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:16:28.721544 12688 tablet_service.cc:1467] Processing CreateTablet for tablet 779c932dfd60445194ee4b0e45879f23 (DEFAULT_TABLE table=test-table [id=c3d84190b3f4474ba3cc886a672dee2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:16:28.721544 12940 tablet_service.cc:1467] Processing CreateTablet for tablet 779c932dfd60445194ee4b0e45879f23 (DEFAULT_TABLE table=test-table [id=c3d84190b3f4474ba3cc886a672dee2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:16:28.722712 13204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 779c932dfd60445194ee4b0e45879f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:16:28.722715 12688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 779c932dfd60445194ee4b0e45879f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:16:28.722715 12940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 779c932dfd60445194ee4b0e45879f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:16:28.729892 13355 tablet_bootstrap.cc:492] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29: Bootstrap starting.
I20260812 03:16:28.731128 13359 tablet_bootstrap.cc:492] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f: Bootstrap starting.
I20260812 03:16:28.730765 13356 tablet_bootstrap.cc:492] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814: Bootstrap starting.
I20260812 03:16:28.732769 13355 tablet_bootstrap.cc:654] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29: Neither blocks nor log segments found. Creating new log.
I20260812 03:16:28.733348 13359 tablet_bootstrap.cc:654] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f: Neither blocks nor log segments found. Creating new log.
I20260812 03:16:28.733891 13356 tablet_bootstrap.cc:654] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814: Neither blocks nor log segments found. Creating new log.
I20260812 03:16:28.734102 13359 log.cc:824] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:28.734104 13355 log.cc:824] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:28.734705 13356 log.cc:824] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:28.736271 13355 tablet_bootstrap.cc:492] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29: No bootstrap required, opened a new log
I20260812 03:16:28.736300 13359 tablet_bootstrap.cc:492] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f: No bootstrap required, opened a new log
I20260812 03:16:28.736486 13355 ts_tablet_manager.cc:1397] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 03:16:28.736488 13359 ts_tablet_manager.cc:1397] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:16:28.736521 13356 tablet_bootstrap.cc:492] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814: No bootstrap required, opened a new log
I20260812 03:16:28.737666 13356 ts_tablet_manager.cc:1397] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 03:16:28.739673 13355 raft_consensus.cc:348] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:28.739928 13355 raft_consensus.cc:374] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:16:28.739984 13355 raft_consensus.cc:729] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fd81512c7924e789ad904a4e8c4eb29, State: Initialized, Role: FOLLOWER
I20260812 03:16:28.740432 13355 consensus_queue.cc:260] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:28.740434 13359 raft_consensus.cc:348] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:28.740697 13359 raft_consensus.cc:374] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:16:28.740754 13359 raft_consensus.cc:729] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54df08210dcd4b84973990752dc3f55f, State: Initialized, Role: FOLLOWER
I20260812 03:16:28.741158 13359 consensus_queue.cc:260] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:28.741303 13319 heartbeater.cc:503] Master 127.16.217.126:39545 was elected leader, sending a full tablet report...
I20260812 03:16:28.741496 13355 ts_tablet_manager.cc:1428] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 03:16:28.742123 13071 heartbeater.cc:503] Master 127.16.217.126:39545 was elected leader, sending a full tablet report...
I20260812 03:16:28.742143 13356 raft_consensus.cc:348] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:28.742415 13356 raft_consensus.cc:374] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:16:28.742478 13356 raft_consensus.cc:729] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f0fa8043f941ae81e2e2ba831c4814, State: Initialized, Role: FOLLOWER
I20260812 03:16:28.742861 13359 ts_tablet_manager.cc:1428] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 03:16:28.742857 13356 consensus_queue.cc:260] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:28.743499 12806 heartbeater.cc:503] Master 127.16.217.126:39545 was elected leader, sending a full tablet report...
I20260812 03:16:28.744040 13356 ts_tablet_manager.cc:1428] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:16:28.813784 13364 raft_consensus.cc:482] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:16:28.814033 13364 raft_consensus.cc:504] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:28.815281 13364 leader_election.cc:288] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54df08210dcd4b84973990752dc3f55f (127.16.217.66:42703), e4f0fa8043f941ae81e2e2ba831c4814 (127.16.217.65:45103)
I20260812 03:16:28.820060 13347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.815697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:37757 (local address 127.16.217.66:42703)
0812 03:16:28.815857 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:28.815865 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:28.815884 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:16:28.815989 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:28.815992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:28.816080 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:28.816186 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:28.816535 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.817257 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.818805 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.819443 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.819556 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:28.819625 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:28.819858 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 03:16:28.819936 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:16:28.820078 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.815532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:42703 (local address 127.16.217.67:37757)
0812 03:16:28.815667 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:28.815685 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:28.815775 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:28.816230 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:28.816234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:28.816257 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:28.816410 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:28.816418 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.817370 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.817376 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:28.818680 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:28.818693 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.819581 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.819585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:28.819707 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:28.819738 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 03:16:28.819832 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 03:16:28.819908 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 03:16:28.820725 13352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.816031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:56325 (local address 127.16.217.65:45103)
0812 03:16:28.816209 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:28.816213 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:28.816340 (+ 127us) server_negotiation.cc:407] Connection header received
0812 03:16:28.816456 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:28.816461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:28.816516 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:28.816589 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:28.816906 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.817626 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.819173 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.819738 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.819870 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:28.819922 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:28.820521 (+ 599us) server_negotiation.cc:299] Negotiation successful
0812 03:16:28.820611 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 03:16:28.820827 12986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "779c932dfd60445194ee4b0e45879f23" candidate_uuid: "0fd81512c7924e789ad904a4e8c4eb29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54df08210dcd4b84973990752dc3f55f" is_pre_election: true
I20260812 03:16:28.821111 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.815793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:45103 (local address 127.16.217.67:56325)
0812 03:16:28.816228 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:28.816248 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:28.816369 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:28.816635 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:28.816638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:28.816659 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:28.816786 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:28.816794 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.817774 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.817778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:28.819044 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:28.819053 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.820262 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.820266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:28.820372 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:28.820396 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:16:28.820905 (+ 509us) client_negotiation.cc:240] Negotiation successful
0812 03:16:28.820980 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":173,"threads_started":1}
I20260812 03:16:28.821146 12986 raft_consensus.cc:2454] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fd81512c7924e789ad904a4e8c4eb29 in term 0.
I20260812 03:16:28.821877 12731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "779c932dfd60445194ee4b0e45879f23" candidate_uuid: "0fd81512c7924e789ad904a4e8c4eb29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" is_pre_election: true
I20260812 03:16:28.821907 13112 leader_election.cc:302] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fd81512c7924e789ad904a4e8c4eb29, 54df08210dcd4b84973990752dc3f55f; no voters:
I20260812 03:16:28.822198 13364 raft_consensus.cc:2781] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:16:28.822186 12731 raft_consensus.cc:2454] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fd81512c7924e789ad904a4e8c4eb29 in term 0.
I20260812 03:16:28.822309 13364 raft_consensus.cc:482] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:16:28.822357 13364 raft_consensus.cc:3037] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:16:28.823680 13364 raft_consensus.cc:504] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:28.824030 13364 leader_election.cc:288] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [CANDIDATE]: Term 1 election: Requested vote from peers 54df08210dcd4b84973990752dc3f55f (127.16.217.66:42703), e4f0fa8043f941ae81e2e2ba831c4814 (127.16.217.65:45103)
I20260812 03:16:28.824539 12986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "779c932dfd60445194ee4b0e45879f23" candidate_uuid: "0fd81512c7924e789ad904a4e8c4eb29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54df08210dcd4b84973990752dc3f55f"
I20260812 03:16:28.824574 12731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "779c932dfd60445194ee4b0e45879f23" candidate_uuid: "0fd81512c7924e789ad904a4e8c4eb29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f0fa8043f941ae81e2e2ba831c4814"
I20260812 03:16:28.824687 12986 raft_consensus.cc:3037] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:16:28.824697 12731 raft_consensus.cc:3037] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:16:28.825929 12731 raft_consensus.cc:2454] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fd81512c7924e789ad904a4e8c4eb29 in term 1.
I20260812 03:16:28.825942 12986 raft_consensus.cc:2454] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fd81512c7924e789ad904a4e8c4eb29 in term 1.
I20260812 03:16:28.826357 13112 leader_election.cc:302] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fd81512c7924e789ad904a4e8c4eb29, 54df08210dcd4b84973990752dc3f55f; no voters:
I20260812 03:16:28.826547 13364 raft_consensus.cc:2781] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:16:28.826807 13364 raft_consensus.cc:686] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [term 1 LEADER]: Becoming Leader. State: Replica: 0fd81512c7924e789ad904a4e8c4eb29, State: Running, Role: LEADER
I20260812 03:16:28.827435 13364 consensus_queue.cc:237] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:28.830507 12465 catalog_manager.cc:5495] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fd81512c7924e789ad904a4e8c4eb29 (127.16.217.67). New cstate: current_term: 1 leader_uuid: "0fd81512c7924e789ad904a4e8c4eb29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } health_report { overall_health: UNKNOWN } } }
I20260812 03:16:28.845721 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.841785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38552 (local address 127.16.217.67:40165)
0812 03:16:28.841925 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:28.841928 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:28.841942 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:16:28.842014 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:28.842016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:28.842086 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:28.842181 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:16:28.842462 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.842947 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.843891 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:28.844208 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:28.844306 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:28.844759 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:16:28.845222 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:16:28.845365 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:28.845481 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 03:16:28.845541 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:16:28.852030 12986 raft_consensus.cc:1261] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f [term 1 FOLLOWER]: Refusing update from remote peer 0fd81512c7924e789ad904a4e8c4eb29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:16:28.852030 12731 raft_consensus.cc:1261] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Refusing update from remote peer 0fd81512c7924e789ad904a4e8c4eb29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:16:28.852759 13364 consensus_queue.cc:1035] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:16:28.852962 13375 consensus_queue.cc:1035] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:16:28.862792 13382 mvcc.cc:205] Tried to move back new op lower bound from 7317522795929645056 to 7317522795839647744. Current Snapshot: MvccSnapshot[applied={T|T < 7317522795929645056}]
I20260812 03:16:28.862373 13383 mvcc.cc:205] Tried to move back new op lower bound from 7317522795929645056 to 7317522795839647744. Current Snapshot: MvccSnapshot[applied={T|T < 7317522795929645056}]
I20260812 03:16:28.870741 12462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:16:28.870944 12462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:16:28.873471 12462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: Sending DeleteTablet for 3 replicas of tablet 779c932dfd60445194ee4b0e45879f23
I20260812 03:16:28.874279 13204 tablet_service.cc:1514] Processing DeleteTablet for tablet 779c932dfd60445194ee4b0e45879f23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:16:28 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:38538
I20260812 03:16:28.874420 12688 tablet_service.cc:1514] Processing DeleteTablet for tablet 779c932dfd60445194ee4b0e45879f23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:16:28 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:52608
I20260812 03:16:28.874781 12940 tablet_service.cc:1514] Processing DeleteTablet for tablet 779c932dfd60445194ee4b0e45879f23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:16:28 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:39356
I20260812 03:16:28.875016 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 12572
I20260812 03:16:28.887195 13399 tablet_replica.cc:330] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29: stopping tablet replica
I20260812 03:16:28.887542 13399 raft_consensus.cc:2229] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:16:28.887989 13399 raft_consensus.cc:2258] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:16:28.891311 13400 tablet_replica.cc:330] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f: stopping tablet replica
I20260812 03:16:28.891685 13400 raft_consensus.cc:2229] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:16:28.892038 13400 raft_consensus.cc:2258] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:16:28.892748 13399 ts_tablet_manager.cc:1905] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:16:28.893834 13400 ts_tablet_manager.cc:1905] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:16:28.896286 13399 ts_tablet_manager.cc:1918] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:16:28.896486 13399 log.cc:1196] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/779c932dfd60445194ee4b0e45879f23
I20260812 03:16:28.896950 13399 ts_tablet_manager.cc:1939] T 779c932dfd60445194ee4b0e45879f23 P 0fd81512c7924e789ad904a4e8c4eb29: Deleting consensus metadata
I20260812 03:16:28.896977 13400 ts_tablet_manager.cc:1918] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:16:28.897065 13400 log.cc:1196] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/779c932dfd60445194ee4b0e45879f23
I20260812 03:16:28.897385 13400 ts_tablet_manager.cc:1939] T 779c932dfd60445194ee4b0e45879f23 P 54df08210dcd4b84973990752dc3f55f: Deleting consensus metadata
I20260812 03:16:28.898547 12441 catalog_manager.cc:4841] TS 0fd81512c7924e789ad904a4e8c4eb29 (127.16.217.67:40165): tablet 779c932dfd60445194ee4b0e45879f23 (table test-table [id=c3d84190b3f4474ba3cc886a672dee2a]) successfully deleted
I20260812 03:16:28.898573 12436 catalog_manager.cc:4841] TS 54df08210dcd4b84973990752dc3f55f (127.16.217.66:42703): tablet 779c932dfd60445194ee4b0e45879f23 (table test-table [id=c3d84190b3f4474ba3cc886a672dee2a]) successfully deleted
W20260812 03:16:28.899832 12436 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.217.65:45103 (error 108)
I20260812 03:16:28.900092 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 12819
I20260812 03:16:28.923398 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:28.923074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:45103 (local address 127.0.0.1:52616)
0812 03:16:28.923216 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:28.923300 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:45103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 03:16:28.923636 12436 catalog_manager.cc:4584] TS e4f0fa8043f941ae81e2e2ba831c4814 (127.16.217.65:45103): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 779c932dfd60445194ee4b0e45879f23: Network error: Client connection negotiation failed: client connection to 127.16.217.65:45103: connect: Connection refused (error 111)
I20260812 03:16:28.928815 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 13083
I20260812 03:16:28.947274 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 12410
I20260812 03:16:48.961802 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:39545
--webserver_interface=127.16.217.126
--webserver_port=42413
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:46323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:39545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:16:49.084177 13404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:49.084488 13404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:49.084566 13404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:49.088617 13404 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 03:16:49.088696 13404 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 03:16:49.088723 13404 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 03:16:49.088752 13404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:16:49.088796 13404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:49.088841 13404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:49.088879 13404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:16:49.088915 13404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:16:49.094122 13404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:46323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:39545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:39545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=42413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:49.095234 13404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:49.096290 13404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:49.110303 13424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:49.110350 13414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:49.110544 13404 server_base.cc:1034] running on GCE node
W20260812 03:16:49.110330 13415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:49.111311 13404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:49.112504 13404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:49.113696 13404 hybrid_clock.cc:648] HybridClock initialized: now 1786504609113687 us; error 34 us; skew 500 ppm
Aug 12 03:16:49 dist-test-slave-7nm7 krb5kdc[12390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504609, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:49.116524 13404 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:16:49.117378 13404 webserver.cc:460] Webserver started at http://127.16.217.126:42413/ using document root <none> and password file <none>
I20260812 03:16:49.117919 13404 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:49.118000 13404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:49.122131 13404 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:16:49.124507 13434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:49.125953 13404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:16:49.126088 13404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "1a246a48a5fd44a78d33e137849e7707"
format_stamp: "Formatted at 2026-08-12 03:16:27 on dist-test-slave-7nm7"
server_key: "c93a6b2b3d7172146120e4503cfb80b1"
server_key_iv: "e91a8d041fcddbe8cfdaa19803619f1d"
server_key_version: "encryptionkey@0"
I20260812 03:16:49.126797 13404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:49.140134 13404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:49.143378 13404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:49.143551 13404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:49.151228 13404 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:39545
I20260812 03:16:49.151623 13520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:39545 every 8 connection(s)
I20260812 03:16:49.152431 13404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:16:49.157492 13521 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:16:49.159469 13521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: Bootstrap starting.
I20260812 03:16:49.160668 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 13404
I20260812 03:16:49.161194 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:45103
--local_ip_for_outbound_sockets=127.16.217.65
--tserver_master_addrs=127.16.217.126:39545
--webserver_port=34249
--webserver_interface=127.16.217.65
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:46323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:16:49.182433 13521 log.cc:824] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:49.193768 13521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:16:49.194288 13521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: Bootstrap complete.
I20260812 03:16:49.199925 13521 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a246a48a5fd44a78d33e137849e7707" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39545 } }
I20260812 03:16:49.200574 13521 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a246a48a5fd44a78d33e137849e7707, State: Initialized, Role: FOLLOWER
I20260812 03:16:49.201328 13521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a246a48a5fd44a78d33e137849e7707" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39545 } }
I20260812 03:16:49.201499 13521 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:16:49.201577 13521 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:16:49.201676 13521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:16:49.204268 13521 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a246a48a5fd44a78d33e137849e7707" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39545 } }
I20260812 03:16:49.204658 13521 leader_election.cc:302] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a246a48a5fd44a78d33e137849e7707; no voters:
I20260812 03:16:49.205039 13521 leader_election.cc:288] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:16:49.205206 13530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:16:49.205493 13530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 2 LEADER]: Becoming Leader. State: Replica: 1a246a48a5fd44a78d33e137849e7707, State: Running, Role: LEADER
I20260812 03:16:49.205801 13530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a246a48a5fd44a78d33e137849e7707" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39545 } }
I20260812 03:16:49.206642 13521 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:16:49.207513 13533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a246a48a5fd44a78d33e137849e7707. Latest consensus state: current_term: 2 leader_uuid: "1a246a48a5fd44a78d33e137849e7707" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a246a48a5fd44a78d33e137849e7707" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39545 } } }
I20260812 03:16:49.207571 13532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1a246a48a5fd44a78d33e137849e7707" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a246a48a5fd44a78d33e137849e7707" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39545 } } }
I20260812 03:16:49.207669 13533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [sys.catalog]: This master's current role is: LEADER
I20260812 03:16:49.207683 13532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [sys.catalog]: This master's current role is: LEADER
I20260812 03:16:49.208231 13545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:16:49.213358 13545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:16:49.214203 13545 catalog_manager.cc:1259] Loaded cluster ID: 2bea95d08bf047dd91d9c6768a3fe5a5
I20260812 03:16:49.214315 13545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:16:49.216063 13545 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:16:49.216671 13545 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: Loaded TSK: 0
I20260812 03:16:49.227213 13545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: Generated new TSK 1
I20260812 03:16:49.227491 13545 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:16:49.316105 13526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:49.316512 13526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:49.316587 13526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:49.322942 13526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:49.323052 13526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:49.323170 13526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:16:49.329516 13526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:46323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:45103
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=34249
--tserver_master_addrs=127.16.217.126:39545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:49.330972 13526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:49.332512 13526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:49.347664 13585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:49.347668 13584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:49.347815 13588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:49.348129 13526 server_base.cc:1034] running on GCE node
I20260812 03:16:49.348748 13526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:49.349601 13526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:49.350951 13526 hybrid_clock.cc:648] HybridClock initialized: now 1786504609350900 us; error 41 us; skew 500 ppm
Aug 12 03:16:49 dist-test-slave-7nm7 krb5kdc[12390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504609, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:49.355474 13526 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:16:49.356878 13526 webserver.cc:460] Webserver started at http://127.16.217.65:34249/ using document root <none> and password file <none>
I20260812 03:16:49.357720 13526 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:49.357854 13526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:49.364074 13526 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:16:49.367224 13599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:49.368896 13526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:16:49.369024 13526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "e4f0fa8043f941ae81e2e2ba831c4814"
format_stamp: "Formatted at 2026-08-12 03:16:28 on dist-test-slave-7nm7"
server_key: "e4fb73f7d157785bbc3be0f79ff92856"
server_key_iv: "9ecf0a21317b38d905140a108ce7236c"
server_key_version: "encryptionkey@0"
I20260812 03:16:49.369736 13526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:49.401144 13526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:49.405349 13526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:49.405521 13526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:49.406103 13526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:16:49.407491 13613 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:16:49.411994 13526 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:16:49.412075 13526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260812 03:16:49.412149 13526 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:16:49.413899 13526 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:16:49.413949 13526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 03:16:49.414386 13613 tablet_bootstrap.cc:492] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814: Bootstrap starting.
I20260812 03:16:49.425781 13526 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:45103
I20260812 03:16:49.425875 13829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:45103 every 8 connection(s)
I20260812 03:16:49.426878 13526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:16:49.432085 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 13526
I20260812 03:16:49.432651 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:42703
--local_ip_for_outbound_sockets=127.16.217.66
--tserver_master_addrs=127.16.217.126:39545
--webserver_port=36211
--webserver_interface=127.16.217.66
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:46323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:16:49.434157 13613 log.cc:824] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:16:49 dist-test-slave-7nm7 krb5kdc[12390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504609, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:16:49.444263 13613 tablet_bootstrap.cc:492] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:16:49.444938 13613 tablet_bootstrap.cc:492] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814: Bootstrap complete.
I20260812 03:16:49.446578 13613 ts_tablet_manager.cc:1397] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814: Time spent bootstrapping tablet: real 0.032s user 0.028s sys 0.004s
I20260812 03:16:49.447191 13837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.429216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:52225)
0812 03:16:49.429899 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:49.429968 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:49.431157 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:49.433006 (+ 1849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:49.433015 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:49.433417 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:49.433887 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:49.433902 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.436273 (+ 2371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.436277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:49.437846 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:49.437858 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.438936 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.438940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:49.439427 (+ 487us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:49.440111 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:49.440128 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:49.441668 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:49.443852 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:49.443857 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:49.443870 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:49.444149 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:49.444603 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:49.444606 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:49.444608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:49.444766 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:49.445405 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:49.445411 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:49.445651 (+ 240us) client_negotiation.cc:769] Sending connection context
0812 03:16:49.445842 (+ 191us) client_negotiation.cc:240] Negotiation successful
0812 03:16:49.446190 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":157,"threads_started":1}
I20260812 03:16:49.447192 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.429227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:52225 (local address 127.16.217.126:39545)
0812 03:16:49.429657 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:49.429678 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:49.430117 (+ 439us) server_negotiation.cc:407] Connection header received
0812 03:16:49.431871 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:49.431905 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:49.432346 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:49.432718 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:49.434106 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.436068 (+ 1962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.438031 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.438757 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.438974 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:49.441883 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:49.441888 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:49.441901 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:49.441930 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:49.443718 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:49.444285 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:49.444290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:49.444294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:49.444471 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:49.444867 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:49.444871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:49.444873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:49.445280 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:49.445417 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:49.445956 (+ 539us) server_negotiation.cc:299] Negotiation successful
0812 03:16:49.446265 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":133,"threads_started":1}
I20260812 03:16:49.449922 13834 heartbeater.cc:347] Connected to a master server at 127.16.217.126:39545
I20260812 03:16:49.450183 13834 heartbeater.cc:464] Registering TS with master...
I20260812 03:16:49.450313 13613 raft_consensus.cc:348] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:49.450778 13834 heartbeater.cc:511] Master 127.16.217.126:39545 requested a full tablet report, sending...
I20260812 03:16:49.450858 13613 raft_consensus.cc:729] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f0fa8043f941ae81e2e2ba831c4814, State: Initialized, Role: FOLLOWER
I20260812 03:16:49.451510 13613 consensus_queue.cc:260] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:49.452636 13613 ts_tablet_manager.cc:1428] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 03:16:49.455298 13467 ts_manager.cc:194] Registered new tserver with Master: e4f0fa8043f941ae81e2e2ba831c4814 (127.16.217.65:45103)
I20260812 03:16:49.458904 13467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:52225
I20260812 03:16:49.461660 13834 heartbeater.cc:503] Master 127.16.217.126:39545 was elected leader, sending a full tablet report...
Aug 12 03:16:49 dist-test-slave-7nm7 krb5kdc[12390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504609, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for kudu/127.16.217.65@KRBTEST.COM
I20260812 03:16:49.469749 13847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.458367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37264 (local address 127.16.217.65:45103)
0812 03:16:49.459094 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:49.459108 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:49.459136 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:16:49.459214 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:49.459220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:49.459420 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:49.459572 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:49.460378 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.461446 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.462714 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.463213 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.463333 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:49.466005 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:49.466015 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:49.466034 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:49.466057 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:49.467715 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:49.468252 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:49.468257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:49.468259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:49.468336 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:49.468648 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:49.468651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:49.468653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:49.468906 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:49.469061 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:49.469412 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 03:16:49.469583 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":126,"threads_started":1}
I20260812 03:16:49.469749 13846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.458250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:45103 (local address 127.0.0.1:37264)
0812 03:16:49.458706 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:49.458759 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:49.458903 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:49.459606 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:49.459621 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:49.460088 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:49.460236 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:49.460245 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.461581 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.461584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:49.462557 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:49.462570 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.463334 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.463338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:49.463448 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:49.464082 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:49.464095 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:49.465844 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:49.467853 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:49.467863 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:49.467869 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:49.468113 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:49.468439 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:49.468443 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:49.468445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:49.468548 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:49.469033 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:49.469040 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:49.469182 (+ 142us) client_negotiation.cc:769] Sending connection context
0812 03:16:49.469328 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 03:16:49.469556 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":120,"threads_started":1}
I20260812 03:16:49.470811 13704 tablet_service.cc:1514] Processing DeleteTablet for tablet 779c932dfd60445194ee4b0e45879f23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:16:28 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:37264
I20260812 03:16:49.471709 13848 tablet_replica.cc:330] stopping tablet replica
I20260812 03:16:49.472029 13848 raft_consensus.cc:2229] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:16:49.472298 13848 raft_consensus.cc:2258] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:16:49.473333 13848 ts_tablet_manager.cc:1905] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:16:49.478717 13848 ts_tablet_manager.cc:1918] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:16:49.478816 13848 log.cc:1196] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/wal/wals/779c932dfd60445194ee4b0e45879f23
I20260812 03:16:49.479141 13848 ts_tablet_manager.cc:1939] T 779c932dfd60445194ee4b0e45879f23 P e4f0fa8043f941ae81e2e2ba831c4814: Deleting consensus metadata
I20260812 03:16:49.480882 13436 catalog_manager.cc:4841] TS e4f0fa8043f941ae81e2e2ba831c4814 (127.16.217.65:45103): tablet 779c932dfd60445194ee4b0e45879f23 (table test-table [id=c3d84190b3f4474ba3cc886a672dee2a]) successfully deleted
W20260812 03:16:49.560896 13839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:49.561187 13839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:49.561259 13839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:49.565428 13839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:49.565524 13839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:49.565634 13839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:16:49.571002 13839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:46323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:42703
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=36211
--tserver_master_addrs=127.16.217.126:39545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:49.572152 13839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:49.573210 13839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:49.583886 13860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:49.583966 13863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:49.583940 13857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:49.583901 13839 server_base.cc:1034] running on GCE node
I20260812 03:16:49.584616 13839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:49.585383 13839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:49.586560 13839 hybrid_clock.cc:648] HybridClock initialized: now 1786504609586531 us; error 52 us; skew 500 ppm
Aug 12 03:16:49 dist-test-slave-7nm7 krb5kdc[12390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504609, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:49.589458 13839 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:16:49.590441 13839 webserver.cc:460] Webserver started at http://127.16.217.66:36211/ using document root <none> and password file <none>
I20260812 03:16:49.590986 13839 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:49.591060 13839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:49.595209 13839 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:16:49.597328 13874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:49.598711 13839 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:16:49.598850 13839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "54df08210dcd4b84973990752dc3f55f"
format_stamp: "Formatted at 2026-08-12 03:16:28 on dist-test-slave-7nm7"
server_key: "6ee30a311a7653e64938c1bff06670b0"
server_key_iv: "c26812957020c25a2c38969ba31c4b7d"
server_key_version: "encryptionkey@0"
I20260812 03:16:49.599398 13839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:49.616422 13839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:49.620558 13839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:49.620723 13839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:49.621280 13839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:16:49.622107 13839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:16:49.622154 13839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:49.622195 13839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:16:49.622210 13839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:49.631793 13839 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:42703
I20260812 03:16:49.631819 14084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:42703 every 8 connection(s)
I20260812 03:16:49.632863 13839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:16:49.634020 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 13839
I20260812 03:16:49.634574 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:40165
--local_ip_for_outbound_sockets=127.16.217.67
--tserver_master_addrs=127.16.217.126:39545
--webserver_port=42005
--webserver_interface=127.16.217.67
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:46323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:16:49 dist-test-slave-7nm7 krb5kdc[12390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504609, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:16:49.648778 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.635059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:56541 (local address 127.16.217.126:39545)
0812 03:16:49.635261 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:49.635269 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:49.636178 (+ 909us) server_negotiation.cc:407] Connection header received
0812 03:16:49.638048 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:49.638051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:49.638107 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:49.638191 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:49.639512 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.640043 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.641336 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.641578 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.641701 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:49.644576 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:49.644579 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:49.644582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:49.644608 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:49.646216 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:49.646967 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:49.646970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:49.646971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:49.647048 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:49.647481 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:49.647484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:49.647485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:49.647685 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:49.647768 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:49.648481 (+ 713us) server_negotiation.cc:299] Negotiation successful
0812 03:16:49.648604 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 03:16:49.649868 14095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.635468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:56541)
0812 03:16:49.635961 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:49.636014 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:49.637763 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:49.638424 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:49.638431 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:49.638845 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:49.639349 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:49.639363 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.640272 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.640276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:49.641196 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:49.641207 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.641750 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.641754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:49.641931 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:49.642634 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:49.642654 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:49.644366 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:49.646388 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:49.646399 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:49.646418 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:49.646854 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:49.647149 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:49.647154 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:49.647157 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:49.647368 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:49.647799 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:49.647809 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:49.648188 (+ 379us) client_negotiation.cc:769] Sending connection context
0812 03:16:49.648429 (+ 241us) client_negotiation.cc:240] Negotiation successful
0812 03:16:49.648817 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":139,"threads_started":1}
I20260812 03:16:49.651257 14087 heartbeater.cc:347] Connected to a master server at 127.16.217.126:39545
I20260812 03:16:49.651516 14087 heartbeater.cc:464] Registering TS with master...
I20260812 03:16:49.652038 14087 heartbeater.cc:511] Master 127.16.217.126:39545 requested a full tablet report, sending...
I20260812 03:16:49.653487 13467 ts_manager.cc:194] Registered new tserver with Master: 54df08210dcd4b84973990752dc3f55f (127.16.217.66:42703)
I20260812 03:16:49.654340 13467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:56541
W20260812 03:16:49.774622 14097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:16:49.774914 14097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:16:49.774962 14097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:16:49.779512 14097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:16:49.779597 14097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:16:49.779662 14097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:16:49.788359 14097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:46323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:40165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=42005
--tserver_master_addrs=127.16.217.126:39545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:16:49.789719 14097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:16:49.791055 14097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:16:49.802520 14117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:49.802569 14113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:16:49.802569 14120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:16:49.802727 14097 server_base.cc:1034] running on GCE node
I20260812 03:16:49.803510 14097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:16:49.804258 14097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:16:49.805449 14097 hybrid_clock.cc:648] HybridClock initialized: now 1786504609805430 us; error 39 us; skew 500 ppm
Aug 12 03:16:49 dist-test-slave-7nm7 krb5kdc[12390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504609, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:16:49.808952 14097 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:16:49.809947 14097 webserver.cc:460] Webserver started at http://127.16.217.67:42005/ using document root <none> and password file <none>
I20260812 03:16:49.810637 14097 fs_manager.cc:359] Metadata directory not provided
I20260812 03:16:49.810746 14097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:16:49.814886 14097 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:16:49.817440 14134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:49.819130 14097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:16:49.819274 14097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "0fd81512c7924e789ad904a4e8c4eb29"
format_stamp: "Formatted at 2026-08-12 03:16:28 on dist-test-slave-7nm7"
server_key: "0971184d74140994b2845cca676b5892"
server_key_iv: "de86c96998fab53b52b02f6c72c41b44"
server_key_version: "encryptionkey@0"
I20260812 03:16:49.819847 14097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:16:49.855289 14097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:16:49.858928 14097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:16:49.859167 14097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:16:49.859833 14097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:16:49.860739 14097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:16:49.860813 14097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:49.860886 14097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:16:49.860929 14097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:16:49.871376 14097 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:40165
I20260812 03:16:49.871397 14350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:40165 every 8 connection(s)
I20260812 03:16:49.872378 14097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:16:49.874917 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 14097
Aug 12 03:16:49 dist-test-slave-7nm7 krb5kdc[12390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504609, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:16:49.884019 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.875571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41696 (local address 127.16.217.126:39545)
0812 03:16:49.876193 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:49.876200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:49.876215 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:16:49.876267 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:49.876269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:49.876310 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:49.876410 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:49.877133 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.877669 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.878605 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.878837 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.878945 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:49.880365 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:49.880369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:49.880371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:49.880394 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:49.882028 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:49.882622 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:49.882625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:49.882627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:49.882713 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:49.882997 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:49.882999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:49.883000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:49.883171 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:49.883360 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:49.883594 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 03:16:49.883781 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"spinlock_wait_cycles":6912,"thread_start_us":100,"threads_started":1}
I20260812 03:16:49.887017 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.874203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:58635 (local address 127.16.217.126:39545)
0812 03:16:49.874381 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:49.874385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:49.875679 (+ 1294us) server_negotiation.cc:407] Connection header received
0812 03:16:49.876645 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:49.876648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:49.876698 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:49.876776 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:49.877871 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.878453 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.879707 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.879942 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.880062 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:49.882987 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:49.882992 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:49.882995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:49.883076 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:49.884928 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:49.885437 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:49.885442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:49.885444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:49.885531 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:49.885925 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:49.885930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:49.885932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:49.886110 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:49.886221 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:49.886757 (+ 536us) server_negotiation.cc:299] Negotiation successful
0812 03:16:49.886867 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"spinlock_wait_cycles":768}
I20260812 03:16:49.887828 14355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.874791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:58635)
0812 03:16:49.875514 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:49.875583 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:49.876442 (+ 859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:49.876935 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:49.876942 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:49.877268 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:16:49.877732 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:49.877744 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.878588 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.878591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:49.879579 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:49.879596 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.880095 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.880100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:49.880265 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:49.880812 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:16:49.880835 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:16:49.882744 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:16:49.885048 (+ 2304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:49.885054 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:49.885069 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:49.885326 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:49.885630 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:16:49.885634 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:16:49.885636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:16:49.885813 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:16:49.886221 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:16:49.886256 (+ 35us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:16:49.886502 (+ 246us) client_negotiation.cc:769] Sending connection context
0812 03:16:49.886702 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 03:16:49.887029 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":148,"threads_started":1}
I20260812 03:16:49.888890 14353 heartbeater.cc:347] Connected to a master server at 127.16.217.126:39545
I20260812 03:16:49.889133 14353 heartbeater.cc:464] Registering TS with master...
I20260812 03:16:49.889619 14353 heartbeater.cc:511] Master 127.16.217.126:39545 requested a full tablet report, sending...
I20260812 03:16:49.890705 13467 ts_manager.cc:194] Registered new tserver with Master: 0fd81512c7924e789ad904a4e8c4eb29 (127.16.217.67:40165)
I20260812 03:16:49.891247 13467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:58635
I20260812 03:16:49.897655 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:16:49.901376 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.898308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41710 (local address 127.16.217.126:39545)
0812 03:16:49.898435 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:49.898439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:49.898641 (+ 202us) server_negotiation.cc:407] Connection header received
0812 03:16:49.898771 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:49.898773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:49.898864 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:49.898942 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:16:49.899150 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.899615 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.900418 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.900626 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.900709 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:49.900964 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:16:49.901090 (+ 126us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 03:16:49.901282 (+ 192us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41710: token expired
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 03:16:49.901455 13836 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41710: token expired
W20260812 03:16:49.901813 13323 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.16.217.126:39545)
I20260812 03:16:49.909417 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.902173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41716 (local address 127.16.217.126:39545)
0812 03:16:49.902326 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:49.902330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:49.902343 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:16:49.902477 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:49.902480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:49.902519 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:49.902625 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:16:49.903231 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.903748 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.904502 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.904740 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.904851 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:49.906181 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:16:49.906186 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:16:49.906188 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:16:49.906213 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:16:49.907793 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:49.908212 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:49.908216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:49.908218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:49.908302 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:16:49.908601 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:16:49.908605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:16:49.908606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:16:49.908780 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:16:49.908919 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:49.909139 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 03:16:49.909260 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:16:49.911871 13467 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41716:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:16:49.915174 13467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:16:49.923585 13704 tablet_service.cc:1467] Processing CreateTablet for tablet 0d0d301f5a6843589469ab3f5c9cb11e (DEFAULT_TABLE table=test-table [id=e42a513745dc46048d6ab8f3fcf2bbb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:16:49.924360 13704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d0d301f5a6843589469ab3f5c9cb11e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:16:49.927364 13613 tablet_bootstrap.cc:492] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814: Bootstrap starting.
I20260812 03:16:49.928337 13613 tablet_bootstrap.cc:654] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814: Neither blocks nor log segments found. Creating new log.
I20260812 03:16:49.928344 13846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.922232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:40165 (local address 127.0.0.1:55784)
0812 03:16:49.922386 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:49.922402 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:49.922489 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:49.923655 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:49.923660 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:49.923687 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:49.923797 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:49.923805 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.925438 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.925441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:49.926707 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:49.926718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.927826 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.927832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:49.927958 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:49.928011 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 03:16:49.928092 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 03:16:49.928181 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 03:16:49.928714 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.922987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48408 (local address 127.16.217.66:42703)
0812 03:16:49.923444 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:49.923458 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:49.923483 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:16:49.923738 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:49.923744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:49.924359 (+ 615us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:49.924664 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:49.924905 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.925800 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.927038 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.927786 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.927896 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:49.928043 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:49.928422 (+ 379us) server_negotiation.cc:299] Negotiation successful
0812 03:16:49.928538 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":262,"threads_started":1}
I20260812 03:16:49.928965 14374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.922510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:42703 (local address 127.0.0.1:48408)
0812 03:16:49.922808 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:49.922823 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:49.922917 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:49.924670 (+ 1753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:49.924674 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:49.924694 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:49.924793 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:49.924800 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.925920 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.925926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:49.926895 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:49.926906 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.928105 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.928108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:49.928187 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:49.928206 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 03:16:49.928271 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 03:16:49.928326 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":124,"threads_started":1}
I20260812 03:16:49.929184 14377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.922456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55784 (local address 127.16.217.67:40165)
0812 03:16:49.922939 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:49.922953 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:49.922979 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:16:49.923134 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:49.923147 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:49.923393 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:49.923594 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:49.923949 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.925299 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.926857 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.927686 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.927800 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:49.927960 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:49.928904 (+ 944us) server_negotiation.cc:299] Negotiation successful
0812 03:16:49.929028 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":172,"threads_started":1}
I20260812 03:16:49.930039 13613 tablet_bootstrap.cc:492] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814: No bootstrap required, opened a new log
I20260812 03:16:49.930140 13613 ts_tablet_manager.cc:1397] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:16:49.930935 13613 raft_consensus.cc:348] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:49.931082 13613 raft_consensus.cc:374] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:16:49.931121 13613 raft_consensus.cc:729] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f0fa8043f941ae81e2e2ba831c4814, State: Initialized, Role: FOLLOWER
I20260812 03:16:49.931538 13613 consensus_queue.cc:260] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:49.932319 13613 ts_tablet_manager.cc:1428] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:16:49.932336 13962 tablet_service.cc:1467] Processing CreateTablet for tablet 0d0d301f5a6843589469ab3f5c9cb11e (DEFAULT_TABLE table=test-table [id=e42a513745dc46048d6ab8f3fcf2bbb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:16:49.932333 14215 tablet_service.cc:1467] Processing CreateTablet for tablet 0d0d301f5a6843589469ab3f5c9cb11e (DEFAULT_TABLE table=test-table [id=e42a513745dc46048d6ab8f3fcf2bbb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:16:49.933339 13962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d0d301f5a6843589469ab3f5c9cb11e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:16:49.933415 14215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d0d301f5a6843589469ab3f5c9cb11e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:16:49.940258 14384 tablet_bootstrap.cc:492] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f: Bootstrap starting.
I20260812 03:16:49.940713 14385 tablet_bootstrap.cc:492] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29: Bootstrap starting.
I20260812 03:16:49.942451 14385 tablet_bootstrap.cc:654] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29: Neither blocks nor log segments found. Creating new log.
I20260812 03:16:49.942802 14384 tablet_bootstrap.cc:654] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f: Neither blocks nor log segments found. Creating new log.
I20260812 03:16:49.943517 14384 log.cc:824] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:49.943567 14385 log.cc:824] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29: Log is configured to *not* fsync() on all Append() calls
I20260812 03:16:49.945389 14384 tablet_bootstrap.cc:492] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f: No bootstrap required, opened a new log
I20260812 03:16:49.945398 14385 tablet_bootstrap.cc:492] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29: No bootstrap required, opened a new log
I20260812 03:16:49.945595 14384 ts_tablet_manager.cc:1397] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:16:49.945598 14385 ts_tablet_manager.cc:1397] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 03:16:49.948508 14385 raft_consensus.cc:348] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:49.948735 14385 raft_consensus.cc:374] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:16:49.948798 14385 raft_consensus.cc:729] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fd81512c7924e789ad904a4e8c4eb29, State: Initialized, Role: FOLLOWER
I20260812 03:16:49.948683 14384 raft_consensus.cc:348] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:49.948989 14384 raft_consensus.cc:374] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:16:49.949142 14384 raft_consensus.cc:729] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54df08210dcd4b84973990752dc3f55f, State: Initialized, Role: FOLLOWER
I20260812 03:16:49.949275 14385 consensus_queue.cc:260] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:49.949635 14384 consensus_queue.cc:260] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:49.949915 14353 heartbeater.cc:503] Master 127.16.217.126:39545 was elected leader, sending a full tablet report...
I20260812 03:16:49.950366 14087 heartbeater.cc:503] Master 127.16.217.126:39545 was elected leader, sending a full tablet report...
I20260812 03:16:49.950631 14384 ts_tablet_manager.cc:1428] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 03:16:49.951391 14385 ts_tablet_manager.cc:1428] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:16:49.994071 14391 raft_consensus.cc:482] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:16:49.994367 14391 raft_consensus.cc:504] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:49.995527 14391 leader_election.cc:288] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fd81512c7924e789ad904a4e8c4eb29 (127.16.217.67:40165), e4f0fa8043f941ae81e2e2ba831c4814 (127.16.217.65:45103)
I20260812 03:16:49.998580 14389 raft_consensus.cc:482] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:16:49.998765 14389 raft_consensus.cc:504] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:50.000327 14095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.995846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:45103 (local address 127.16.217.66:57049)
0812 03:16:49.996004 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:49.996021 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:49.996134 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:49.996629 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:49.996632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:49.996669 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:49.996830 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:49.996838 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.997908 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.997911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:49.999015 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:49.999027 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.999868 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.999872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:49.999976 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:49.999998 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:16:50.000098 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 03:16:50.000178 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":47,"spinlock_wait_cycles":2944}
I20260812 03:16:50.000327 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.995990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:57049 (local address 127.16.217.65:45103)
0812 03:16:49.996322 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:49.996326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:49.996339 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:16:49.996412 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:49.996415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:49.996488 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:49.996603 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:49.996952 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.997738 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.999174 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.999732 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.999872 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:50.000018 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:50.000114 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 03:16:50.000188 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":127,"threads_started":1}
I20260812 03:16:50.000356 14389 leader_election.cc:288] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54df08210dcd4b84973990752dc3f55f (127.16.217.66:42703), e4f0fa8043f941ae81e2e2ba831c4814 (127.16.217.65:45103)
I20260812 03:16:50.000985 14377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.995977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:42143 (local address 127.16.217.67:40165)
0812 03:16:49.996127 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:49.996131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:49.996412 (+ 281us) server_negotiation.cc:407] Connection header received
0812 03:16:49.996470 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:49.996472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:49.996515 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:49.996602 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:49.996952 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.997737 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.999405 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:50.000233 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:50.000351 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:50.000409 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:50.000788 (+ 379us) server_negotiation.cc:299] Negotiation successful
0812 03:16:50.000873 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:16:50.001147 13741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d0d301f5a6843589469ab3f5c9cb11e" candidate_uuid: "54df08210dcd4b84973990752dc3f55f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" is_pre_election: true
I20260812 03:16:50.001359 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:49.995846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:40165 (local address 127.16.217.66:42143)
0812 03:16:49.996324 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:49.996335 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:49.996409 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:49.996629 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:49.996632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:49.996678 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:49.996831 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:49.996838 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:49.997921 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:49.997928 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:49.999262 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:49.999273 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:50.000530 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:50.000537 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:50.000642 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:50.000667 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:16:50.001105 (+ 438us) client_negotiation.cc:240] Negotiation successful
0812 03:16:50.001203 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":154,"spinlock_wait_cycles":49280,"thread_start_us":155,"threads_started":1}
I20260812 03:16:50.001447 13741 raft_consensus.cc:2454] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54df08210dcd4b84973990752dc3f55f in term 0.
I20260812 03:16:50.001880 14264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d0d301f5a6843589469ab3f5c9cb11e" candidate_uuid: "54df08210dcd4b84973990752dc3f55f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fd81512c7924e789ad904a4e8c4eb29" is_pre_election: true
I20260812 03:16:50.001963 13876 leader_election.cc:302] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54df08210dcd4b84973990752dc3f55f, e4f0fa8043f941ae81e2e2ba831c4814; no voters:
I20260812 03:16:50.002071 14264 raft_consensus.cc:2454] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54df08210dcd4b84973990752dc3f55f in term 0.
I20260812 03:16:50.002342 14391 raft_consensus.cc:2781] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:16:50.002501 14391 raft_consensus.cc:482] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:16:50.002610 14391 raft_consensus.cc:3037] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:16:50.004225 14391 raft_consensus.cc:504] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:50.004892 14264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d0d301f5a6843589469ab3f5c9cb11e" candidate_uuid: "54df08210dcd4b84973990752dc3f55f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fd81512c7924e789ad904a4e8c4eb29"
I20260812 03:16:50.004926 13741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d0d301f5a6843589469ab3f5c9cb11e" candidate_uuid: "54df08210dcd4b84973990752dc3f55f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f0fa8043f941ae81e2e2ba831c4814"
I20260812 03:16:50.005004 14264 raft_consensus.cc:3037] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:16:50.005043 13741 raft_consensus.cc:3037] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:16:50.005708 14355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:50.000647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:42703 (local address 127.16.217.67:54711)
0812 03:16:50.000952 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:50.000967 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:50.001047 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:50.002022 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:50.002024 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:50.002043 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:50.002144 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:50.002150 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:50.003363 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:50.003367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:50.004309 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:50.004319 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:50.005291 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:50.005296 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:50.005414 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:50.005449 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 03:16:50.005527 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:16:50.005597 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225}
I20260812 03:16:50.006021 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:50.001305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:54711 (local address 127.16.217.66:42703)
0812 03:16:50.001742 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:50.001749 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:50.001763 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:16:50.001804 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:50.001808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:50.001851 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:50.001919 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:50.002595 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:50.003246 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:50.004475 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:50.005042 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:50.005159 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:50.005227 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:50.005847 (+ 620us) server_negotiation.cc:299] Negotiation successful
0812 03:16:50.005918 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354}
I20260812 03:16:50.006580 14264 raft_consensus.cc:2454] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54df08210dcd4b84973990752dc3f55f in term 1.
I20260812 03:16:50.006582 13741 raft_consensus.cc:2454] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54df08210dcd4b84973990752dc3f55f in term 1.
I20260812 03:16:50.006825 13996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d0d301f5a6843589469ab3f5c9cb11e" candidate_uuid: "0fd81512c7924e789ad904a4e8c4eb29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54df08210dcd4b84973990752dc3f55f" is_pre_election: true
I20260812 03:16:50.006888 13880 leader_election.cc:302] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fd81512c7924e789ad904a4e8c4eb29, 54df08210dcd4b84973990752dc3f55f; no voters:
I20260812 03:16:50.007071 14391 leader_election.cc:288] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [CANDIDATE]: Term 1 election: Requested vote from peers 0fd81512c7924e789ad904a4e8c4eb29 (127.16.217.67:40165), e4f0fa8043f941ae81e2e2ba831c4814 (127.16.217.65:45103)
I20260812 03:16:50.007112 13996 raft_consensus.cc:2379] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0fd81512c7924e789ad904a4e8c4eb29 in current term 1: Already voted for candidate 54df08210dcd4b84973990752dc3f55f in this term.
I20260812 03:16:50.007284 14402 raft_consensus.cc:2781] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:16:50.007344 14398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:50.000899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:45103 (local address 127.16.217.67:57149)
0812 03:16:50.001660 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:50.001686 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 03:16:50.001775 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:16:50.002788 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:16:50.002794 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:16:50.002810 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:16:50.002905 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:50.002909 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:50.003749 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:50.003752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:50.005028 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:16:50.005041 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:16:50.005999 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:16:50.006003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:16:50.006098 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:50.006121 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:16:50.006203 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:16:50.006523 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":146,"threads_started":1}
I20260812 03:16:50.007540 14402 raft_consensus.cc:686] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [term 1 LEADER]: Becoming Leader. State: Replica: 54df08210dcd4b84973990752dc3f55f, State: Running, Role: LEADER
I20260812 03:16:50.007333 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:50.001394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:57149 (local address 127.16.217.65:45103)
0812 03:16:50.002493 (+ 1099us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:50.002497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:50.002509 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:16:50.002546 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:50.002549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:50.002586 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:50.002655 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:16:50.003019 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:50.003621 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:50.005189 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:50.005759 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:50.005871 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:50.005916 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:50.007041 (+ 1125us) server_negotiation.cc:299] Negotiation successful
0812 03:16:50.007216 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":986}
I20260812 03:16:50.008085 13741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d0d301f5a6843589469ab3f5c9cb11e" candidate_uuid: "0fd81512c7924e789ad904a4e8c4eb29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" is_pre_election: true
I20260812 03:16:50.008170 14402 consensus_queue.cc:237] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:16:50.008293 13741 raft_consensus.cc:2379] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0fd81512c7924e789ad904a4e8c4eb29 in current term 1: Already voted for candidate 54df08210dcd4b84973990752dc3f55f in this term.
I20260812 03:16:50.008749 14136 leader_election.cc:302] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fd81512c7924e789ad904a4e8c4eb29; no voters: 54df08210dcd4b84973990752dc3f55f, e4f0fa8043f941ae81e2e2ba831c4814
I20260812 03:16:50.009199 14389 raft_consensus.cc:2726] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 03:16:50.011722 13467 catalog_manager.cc:5495] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f reported cstate change: term changed from 0 to 1, leader changed from <none> to 54df08210dcd4b84973990752dc3f55f (127.16.217.66). New cstate: current_term: 1 leader_uuid: "54df08210dcd4b84973990752dc3f55f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } health_report { overall_health: UNKNOWN } } }
I20260812 03:16:50.029860 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:50.026450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48416 (local address 127.16.217.66:42703)
0812 03:16:50.026603 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 03:16:50.026607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:16:50.026627 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:16:50.026753 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:16:50.026756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:16:50.026816 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:16:50.026911 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:16:50.027083 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:50.027578 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:50.028373 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:16:50.028600 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:16:50.028692 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:16:50.028979 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:16:50.029383 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:16:50.029497 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:16:50.029613 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 03:16:50.029662 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:16:50.036886 13741 raft_consensus.cc:1261] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Refusing update from remote peer 54df08210dcd4b84973990752dc3f55f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:16:50.037009 14264 raft_consensus.cc:1261] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29 [term 1 FOLLOWER]: Refusing update from remote peer 54df08210dcd4b84973990752dc3f55f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:16:50.037560 14402 consensus_queue.cc:1035] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:16:50.037791 14404 consensus_queue.cc:1035] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:16:50.044996 14410 mvcc.cc:205] Tried to move back new op lower bound from 7317522882703429632 to 7317522882596683776. Current Snapshot: MvccSnapshot[applied={T|T < 7317522882703429632}]
I20260812 03:16:50.054217 13467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:16:50.054440 13467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:16:50.056993 13467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: Sending DeleteTablet for 3 replicas of tablet 0d0d301f5a6843589469ab3f5c9cb11e
I20260812 03:16:50.057857 14215 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d0d301f5a6843589469ab3f5c9cb11e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:16:50 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:55784
I20260812 03:16:50.058128 13962 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d0d301f5a6843589469ab3f5c9cb11e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:16:50 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:48408
I20260812 03:16:50.058598 13704 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d0d301f5a6843589469ab3f5c9cb11e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:16:50 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:37264
I20260812 03:16:50.058692 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 13526
I20260812 03:16:50.058732 14427 tablet_replica.cc:330] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f: stopping tablet replica
I20260812 03:16:50.070467 14424 tablet_replica.cc:330] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29: stopping tablet replica
I20260812 03:16:50.070940 14424 raft_consensus.cc:2229] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:16:50.070925 14427 raft_consensus.cc:2229] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:16:50.071445 14424 raft_consensus.cc:2258] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:16:50.071512 14427 raft_consensus.cc:2258] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:16:50.073633 14427 ts_tablet_manager.cc:1905] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:16:50.076864 14427 ts_tablet_manager.cc:1918] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:16:50.076977 14427 log.cc:1196] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/0d0d301f5a6843589469ab3f5c9cb11e
I20260812 03:16:50.077325 14427 ts_tablet_manager.cc:1939] T 0d0d301f5a6843589469ab3f5c9cb11e P 54df08210dcd4b84973990752dc3f55f: Deleting consensus metadata
I20260812 03:16:50.078716 13436 catalog_manager.cc:4841] TS 54df08210dcd4b84973990752dc3f55f (127.16.217.66:42703): tablet 0d0d301f5a6843589469ab3f5c9cb11e (table test-table [id=e42a513745dc46048d6ab8f3fcf2bbb7]) successfully deleted
I20260812 03:16:50.078923 14424 ts_tablet_manager.cc:1905] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:16:50.081507 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 13839
W20260812 03:16:50.094972 13436 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.217.65:45103 (error 108)
I20260812 03:16:50.096459 14374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:16:50.096117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:45103 (local address 127.0.0.1:37274)
0812 03:16:50.096280 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 03:16:50.096363 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:45103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 03:16:50.096902 14424 ts_tablet_manager.cc:1918] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:16:50.097065 14424 log.cc:1196] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/0d0d301f5a6843589469ab3f5c9cb11e
W20260812 03:16:50.096748 13436 catalog_manager.cc:4584] TS e4f0fa8043f941ae81e2e2ba831c4814 (127.16.217.65:45103): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0d0d301f5a6843589469ab3f5c9cb11e: Network error: Client connection negotiation failed: client connection to 127.16.217.65:45103: connect: Connection refused (error 111)
I20260812 03:16:50.097847 14424 ts_tablet_manager.cc:1939] T 0d0d301f5a6843589469ab3f5c9cb11e P 0fd81512c7924e789ad904a4e8c4eb29: Deleting consensus metadata
I20260812 03:16:50.099259 13440 catalog_manager.cc:4841] TS 0fd81512c7924e789ad904a4e8c4eb29 (127.16.217.67:40165): tablet 0d0d301f5a6843589469ab3f5c9cb11e (table test-table [id=e42a513745dc46048d6ab8f3fcf2bbb7]) successfully deleted
I20260812 03:16:50.102752 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 14097
I20260812 03:16:50.120487 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 13404
I20260812 03:17:10.142077 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:39545
--webserver_interface=127.16.217.126
--webserver_port=42413
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:46323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:39545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:10.268059 14432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:10.268422 14432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:10.268525 14432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:10.272691 14432 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 03:17:10.272780 14432 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 03:17:10.272812 14432 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 03:17:10.272847 14432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:10.272889 14432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:10.272929 14432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:10.272961 14432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:10.272996 14432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:10.279076 14432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:46323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:39545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:39545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=42413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:10.280283 14432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:10.281353 14432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:17:10.291553 14432 server_base.cc:1034] running on GCE node
W20260812 03:17:10.291468 14446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:10.291464 14444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:10.291525 14443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:10.292465 14432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:10.293579 14432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:10.294747 14432 hybrid_clock.cc:648] HybridClock initialized: now 1786504630294731 us; error 32 us; skew 500 ppm
Aug 12 03:17:10 dist-test-slave-7nm7 krb5kdc[12390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504630, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:10.297713 14432 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:17:10.298887 14432 webserver.cc:460] Webserver started at http://127.16.217.126:42413/ using document root <none> and password file <none>
I20260812 03:17:10.299607 14432 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:10.299679 14432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:10.303695 14432 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:17:10.305925 14458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:10.307444 14432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:17:10.307559 14432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "1a246a48a5fd44a78d33e137849e7707"
format_stamp: "Formatted at 2026-08-12 03:16:27 on dist-test-slave-7nm7"
server_key: "c93a6b2b3d7172146120e4503cfb80b1"
server_key_iv: "e91a8d041fcddbe8cfdaa19803619f1d"
server_key_version: "encryptionkey@0"
I20260812 03:17:10.308112 14432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:10.318924 14432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:10.322209 14432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:10.322459 14432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:10.329788 14432 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:39545
I20260812 03:17:10.330134 14558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:39545 every 8 connection(s)
I20260812 03:17:10.331038 14432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:17:10.335636 14559 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:17:10.337322 14559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: Bootstrap starting.
I20260812 03:17:10.341382 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 14432
I20260812 03:17:10.342146 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:45103
--local_ip_for_outbound_sockets=127.16.217.65
--tserver_master_addrs=127.16.217.126:39545
--webserver_port=34249
--webserver_interface=127.16.217.65
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:46323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:17:10.357434 14559 log.cc:824] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:10.369318 14559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:17:10.369766 14559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: Bootstrap complete.
I20260812 03:17:10.374004 14559 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a246a48a5fd44a78d33e137849e7707" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39545 } }
I20260812 03:17:10.374462 14559 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a246a48a5fd44a78d33e137849e7707, State: Initialized, Role: FOLLOWER
I20260812 03:17:10.375020 14559 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a246a48a5fd44a78d33e137849e7707" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39545 } }
I20260812 03:17:10.375144 14559 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:17:10.375237 14559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:17:10.375353 14559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 2 FOLLOWER]: Advancing to term 3
I20260812 03:17:10.377550 14559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a246a48a5fd44a78d33e137849e7707" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39545 } }
I20260812 03:17:10.377916 14559 leader_election.cc:302] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a246a48a5fd44a78d33e137849e7707; no voters:
I20260812 03:17:10.378413 14559 leader_election.cc:288] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 03:17:10.378473 14564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 03:17:10.378741 14564 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [term 3 LEADER]: Becoming Leader. State: Replica: 1a246a48a5fd44a78d33e137849e7707, State: Running, Role: LEADER
I20260812 03:17:10.379101 14564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a246a48a5fd44a78d33e137849e7707" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39545 } }
I20260812 03:17:10.379738 14559 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:17:10.380488 14565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1a246a48a5fd44a78d33e137849e7707" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a246a48a5fd44a78d33e137849e7707" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39545 } } }
I20260812 03:17:10.380631 14565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:10.381016 14567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a246a48a5fd44a78d33e137849e7707. Latest consensus state: current_term: 3 leader_uuid: "1a246a48a5fd44a78d33e137849e7707" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a246a48a5fd44a78d33e137849e7707" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39545 } } }
I20260812 03:17:10.381174 14567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:10.381484 14575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:17:10.387379 14575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:17:10.388144 14575 catalog_manager.cc:1259] Loaded cluster ID: 2bea95d08bf047dd91d9c6768a3fe5a5
I20260812 03:17:10.388213 14575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:17:10.390817 14575 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:17:10.391610 14575 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: Loaded TSK: 0
I20260812 03:17:10.391703 14575 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1a246a48a5fd44a78d33e137849e7707: Loaded TSK: 1
F20260812 03:17:10.392541 14575 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8e9a7e99ad google::LogMessage::Fail() at ??:0
@ 0x7f8e9a7ec88a google::LogMessage::SendToLog() at ??:0
@ 0x7f8e9a7e94de google::LogMessage::Flush() at ??:0
@ 0x7f8e9a7eaa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8e9d4ad18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f8e9d4a5f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f8ea514e7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f8ea512899c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f8ea5129178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f8ea516adf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f8ea518982b std::function<>::operator()() at ??:0
@ 0x7f8ea5128b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f8ea5129f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8ea5126579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f8ea516a640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8e9ca3b3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8e9ca3bc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f8e9ca3d3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8e9ca2c512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8e9e0706db start_thread at ??:0
@ 0x7f8e992e761f clone at ??:0
W20260812 03:17:10.509690 14562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:10.510113 14562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:10.510188 14562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:10.517931 14562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:10.518050 14562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:10.518167 14562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:17:10.528997 14562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:46323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:45103
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=34249
--tserver_master_addrs=127.16.217.126:39545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:10.530618 14562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:10.532099 14562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:10.547443 14645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:10.547458 14644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:10.547441 14647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:10.548110 14562 server_base.cc:1034] running on GCE node
I20260812 03:17:10.548585 14562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:10.549342 14562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:10.550586 14562 hybrid_clock.cc:648] HybridClock initialized: now 1786504630550550 us; error 37 us; skew 500 ppm
Aug 12 03:17:10 dist-test-slave-7nm7 krb5kdc[12390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504630, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:10.559850 14562 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:17:10.561017 14562 webserver.cc:460] Webserver started at http://127.16.217.65:34249/ using document root <none> and password file <none>
I20260812 03:17:10.561678 14562 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:10.561745 14562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:10.566856 14562 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:17:10.569566 14661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:10.571713 14562 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:17:10.571884 14562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "e4f0fa8043f941ae81e2e2ba831c4814"
format_stamp: "Formatted at 2026-08-12 03:16:28 on dist-test-slave-7nm7"
server_key: "e4fb73f7d157785bbc3be0f79ff92856"
server_key_iv: "9ecf0a21317b38d905140a108ce7236c"
server_key_version: "encryptionkey@0"
I20260812 03:17:10.572511 14562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:10.594789 14562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:10.598667 14562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:10.598892 14562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:10.599570 14562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:10.600803 14675 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:17:10.607113 14562 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:17:10.607242 14562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260812 03:17:10.607331 14562 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:17:10.609148 14562 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:17:10.609232 14562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 03:17:10.609550 14675 tablet_bootstrap.cc:492] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814: Bootstrap starting.
I20260812 03:17:10.621078 14562 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:45103
I20260812 03:17:10.621352 14891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:45103 every 8 connection(s)
I20260812 03:17:10.622781 14562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:17:10.632943 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 14562
I20260812 03:17:10.633536 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:42703
--local_ip_for_outbound_sockets=127.16.217.66
--tserver_master_addrs=127.16.217.126:39545
--webserver_port=36211
--webserver_interface=127.16.217.66
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:46323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:17:10.634189 14675 log.cc:824] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:10.644939 14675 tablet_bootstrap.cc:492] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:17:10.645454 14675 tablet_bootstrap.cc:492] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814: Bootstrap complete.
I20260812 03:17:10.647058 14675 ts_tablet_manager.cc:1397] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814: Time spent bootstrapping tablet: real 0.038s user 0.022s sys 0.014s
I20260812 03:17:10.650488 14675 raft_consensus.cc:348] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:17:10.651050 14675 raft_consensus.cc:729] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f0fa8043f941ae81e2e2ba831c4814, State: Initialized, Role: FOLLOWER
I20260812 03:17:10.651675 14675 consensus_queue.cc:260] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:17:10.652813 14675 ts_tablet_manager.cc:1428] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
W20260812 03:17:10.704687 14895 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:17:10.625418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:44843)
0812 03:17:10.625995 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:10.626055 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:10.627155 (+ 1100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:10.703818 (+ 76663us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":176,"threads_started":1}
W20260812 03:17:10.705343 14892 heartbeater.cc:644] Failed to heartbeat to 127.16.217.126:39545 (0 consecutive failures): Network error: Failed to ping master at 127.16.217.126:39545: Client connection negotiation failed: client connection to 127.16.217.126:39545: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 03:17:10.706211 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:10.705927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:51373)
0812 03:17:10.706073 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:10.706149 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 03:17:10.706902 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:10.706704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:51181)
0812 03:17:10.706808 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:10.706852 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 03:17:10.707183 14892 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:17:10.765961 14900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:10.766359 14900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:10.766455 14900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:10.771231 14900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:10.771311 14900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:10.771409 14900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:17:10.777577 14900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:46323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:42703
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=36211
--tserver_master_addrs=127.16.217.126:39545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:10.778882 14900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:10.779968 14900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:10.792016 14912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:10.792174 14900 server_base.cc:1034] running on GCE node
W20260812 03:17:10.792025 14913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:10.792024 14915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:10.792906 14900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:10.793594 14900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:10.794790 14900 hybrid_clock.cc:648] HybridClock initialized: now 1786504630794773 us; error 28 us; skew 500 ppm
Aug 12 03:17:10 dist-test-slave-7nm7 krb5kdc[12390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504630, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:10.797737 14900 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:17:10.798782 14900 webserver.cc:460] Webserver started at http://127.16.217.66:36211/ using document root <none> and password file <none>
I20260812 03:17:10.799326 14900 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:10.799386 14900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:10.803791 14900 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:17:10.806430 14930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:10.807736 14900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:17:10.807889 14900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "54df08210dcd4b84973990752dc3f55f"
format_stamp: "Formatted at 2026-08-12 03:16:28 on dist-test-slave-7nm7"
server_key: "6ee30a311a7653e64938c1bff06670b0"
server_key_iv: "c26812957020c25a2c38969ba31c4b7d"
server_key_version: "encryptionkey@0"
I20260812 03:17:10.808437 14900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:10.832185 14900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:10.835691 14900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:10.835860 14900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:10.836449 14900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:10.837311 14900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:10.837360 14900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:10.837401 14900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:10.837417 14900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:10.847203 14900 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:42703
I20260812 03:17:10.847409 15125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:42703 every 8 connection(s)
I20260812 03:17:10.848273 14900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:17:10.848809 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 14900
I20260812 03:17:10.849427 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:40165
--local_ip_for_outbound_sockets=127.16.217.67
--tserver_master_addrs=127.16.217.126:39545
--webserver_port=42005
--webserver_interface=127.16.217.67
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:46323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:17:10.852320 15129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:10.850549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:57875)
0812 03:17:10.851407 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:10.851576 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":117,"threads_started":1}
W20260812 03:17:10.852818 15126 heartbeater.cc:644] Failed to heartbeat to 127.16.217.126:39545 (0 consecutive failures): Network error: Failed to ping master at 127.16.217.126:39545: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
I20260812 03:17:10.853714 15129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:10.853261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:45899)
0812 03:17:10.853537 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:10.853624 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260812 03:17:10.854691 15129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:10.854413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:48663)
0812 03:17:10.854542 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:10.854613 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 03:17:10.854930 15126 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:17:10.985057 15133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:10.985375 15133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:10.985422 15133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:10.989696 15133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:10.989759 15133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:10.989866 15133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:17:10.996512 15133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:46323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:40165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=42005
--tserver_master_addrs=127.16.217.126:39545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:10.998515 15133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:11.000129 15133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:17:11.010999 15133 server_base.cc:1034] running on GCE node
W20260812 03:17:11.010946 15148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:11.010946 15147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:11.010946 15153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:11.011626 15133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:11.012548 15133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:11.013767 15133 hybrid_clock.cc:648] HybridClock initialized: now 1786504631013755 us; error 32 us; skew 500 ppm
Aug 12 03:17:11 dist-test-slave-7nm7 krb5kdc[12390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504631, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:11.016813 15133 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:17:11.017995 15133 webserver.cc:460] Webserver started at http://127.16.217.67:42005/ using document root <none> and password file <none>
I20260812 03:17:11.018894 15133 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:11.018965 15133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:11.023417 15133 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260812 03:17:11.025820 15166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:11.027348 15133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:17:11.027500 15133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "0fd81512c7924e789ad904a4e8c4eb29"
format_stamp: "Formatted at 2026-08-12 03:16:28 on dist-test-slave-7nm7"
server_key: "0971184d74140994b2845cca676b5892"
server_key_iv: "de86c96998fab53b52b02f6c72c41b44"
server_key_version: "encryptionkey@0"
I20260812 03:17:11.028083 15133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:11.075184 15133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:11.079713 15133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:11.079972 15133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:11.080662 15133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:11.081698 15133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:11.081780 15133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:11.081857 15133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:11.081907 15133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:11.092568 15133 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:40165
I20260812 03:17:11.092617 15378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:40165 every 8 connection(s)
I20260812 03:17:11.094051 15133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:17:11.098587 15387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:11.096520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:55683)
0812 03:17:11.097479 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:11.097598 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":751,"thread_start_us":555,"threads_started":1}
W20260812 03:17:11.099166 15380 heartbeater.cc:644] Failed to heartbeat to 127.16.217.126:39545 (0 consecutive failures): Network error: Failed to ping master at 127.16.217.126:39545: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
I20260812 03:17:11.099884 15387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:11.099639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:33033)
0812 03:17:11.099763 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:11.099822 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 03:17:11.100706 15387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:11.100398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:45185)
0812 03:17:11.100543 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:11.100630 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 03:17:11.100917 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 15133
W20260812 03:17:11.100971 15380 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 03:17:11.708869 15391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:11.707999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:38067)
0812 03:17:11.708505 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:11.708609 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":96,"threads_started":1}
I20260812 03:17:11.855930 15392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:11.855528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:56341)
0812 03:17:11.855768 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:11.855828 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":74,"threads_started":1}
I20260812 03:17:12.102041 15393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:12.101603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:50647)
0812 03:17:12.101868 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:12.101926 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":82,"threads_started":1}
I20260812 03:17:12.383661 15395 raft_consensus.cc:482] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:17:12.383849 15395 raft_consensus.cc:504] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:17:12.384955 15395 leader_election.cc:288] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fd81512c7924e789ad904a4e8c4eb29 (127.16.217.67:40165), 54df08210dcd4b84973990752dc3f55f (127.16.217.66:42703)
Aug 12 03:17:12 dist-test-slave-7nm7 krb5kdc[12390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504630, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.66@KRBTEST.COM
Aug 12 03:17:12 dist-test-slave-7nm7 krb5kdc[12390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504630, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.67@KRBTEST.COM
I20260812 03:17:12.399205 15403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:12.385158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:40165 (local address 127.16.217.65:55203)
0812 03:17:12.385429 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:12.385444 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:12.385580 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:12.387230 (+ 1650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:12.387236 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:12.387781 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:12.388265 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:12.388286 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:12.389546 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:12.389552 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:12.390523 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:12.390533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:12.391309 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:12.391313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:12.391448 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:12.392221 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:17:12.392244 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:12.394067 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:12.396697 (+ 2630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:12.396702 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:12.396707 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:12.396914 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:12.397280 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:12.397284 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:12.397286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:12.397375 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:12.398100 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:12.398104 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:12.398472 (+ 368us) client_negotiation.cc:769] Sending connection context
0812 03:17:12.398616 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 03:17:12.398980 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":186,"spinlock_wait_cycles":2816,"thread_start_us":76,"threads_started":1}
I20260812 03:17:12.399217 15404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:12.385298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:55203 (local address 127.16.217.67:40165)
0812 03:17:12.385719 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:12.385740 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:12.385776 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:17:12.386526 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:12.386532 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:12.386867 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:12.387169 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:12.388452 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:12.389418 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:12.390700 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:12.391171 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:12.391343 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:12.394307 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:12.394313 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:12.394324 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:12.394352 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:12.396551 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:12.397033 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:12.397046 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:12.397052 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:12.397179 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:12.397470 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:12.397473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:12.397475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:12.397968 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:12.398130 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:12.398764 (+ 634us) server_negotiation.cc:299] Negotiation successful
0812 03:17:12.399034 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":61,"threads_started":1}
I20260812 03:17:12.399252 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:12.385460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:60671 (local address 127.16.217.66:42703)
0812 03:17:12.385821 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:12.385831 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:12.385861 (+ 30us) server_negotiation.cc:407] Connection header received
0812 03:17:12.386527 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:12.386533 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:12.386867 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:12.387170 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:12.388435 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:12.389379 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:12.390644 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:12.391101 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:12.391280 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:12.394019 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:12.394024 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:12.394036 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:12.394065 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:12.395929 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:12.396812 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:12.396817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:12.396821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:12.396939 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:12.397400 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:12.397404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:12.397405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:12.397850 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:12.398013 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:12.398739 (+ 726us) server_negotiation.cc:299] Negotiation successful
0812 03:17:12.399031 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":91,"threads_started":1}
I20260812 03:17:12.399233 15398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:12.385189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:42703 (local address 127.16.217.65:60671)
0812 03:17:12.385519 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:12.385532 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:12.385594 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:12.387230 (+ 1636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:12.387236 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:12.387780 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:12.388262 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:12.388277 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:12.389525 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:12.389533 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:12.390520 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:12.390530 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:12.391230 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:12.391234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:12.391439 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:12.392301 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:17:12.392317 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:12.393793 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:12.396071 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:12.396077 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:12.396090 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:12.396640 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:12.397044 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:12.397048 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:12.397050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:12.397263 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:12.397977 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:12.397983 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:12.398389 (+ 406us) client_negotiation.cc:769] Sending connection context
0812 03:17:12.398616 (+ 227us) client_negotiation.cc:240] Negotiation successful
0812 03:17:12.399072 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":198,"spinlock_wait_cycles":17920,"thread_start_us":75,"threads_started":1}
W20260812 03:17:12.402081 14663 leader_election.cc:341] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 54df08210dcd4b84973990752dc3f55f (127.16.217.66:42703): Not found: Tablet not found: 0d0d301f5a6843589469ab3f5c9cb11e
W20260812 03:17:12.402308 14671 leader_election.cc:341] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fd81512c7924e789ad904a4e8c4eb29 (127.16.217.67:40165): Not found: Tablet not found: 0d0d301f5a6843589469ab3f5c9cb11e
I20260812 03:17:12.402438 14671 leader_election.cc:302] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4f0fa8043f941ae81e2e2ba831c4814; no voters: 0fd81512c7924e789ad904a4e8c4eb29, 54df08210dcd4b84973990752dc3f55f
I20260812 03:17:12.402740 15395 raft_consensus.cc:2726] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:17:12.709985 15403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:12.709706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:37963)
0812 03:17:12.709839 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:12.709896 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 03:17:12.857729 15411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:12.857085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:58067)
0812 03:17:12.857531 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:12.857603 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":115,"threads_started":1}
I20260812 03:17:13.103374 15412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:13.102894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:38319)
0812 03:17:13.103182 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:13.103236 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":107,"threads_started":1}
I20260812 03:17:13.711493 15415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:13.710967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:51281)
0812 03:17:13.711334 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:13.711395 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":160,"threads_started":1}
I20260812 03:17:13.859054 15418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:13.858647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:57997)
0812 03:17:13.858888 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:13.858953 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":76,"threads_started":1}
I20260812 03:17:14.104811 15419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:14.104353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:44237)
0812 03:17:14.104636 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:14.104694 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":83,"threads_started":1}
I20260812 03:17:14.137912 15421 raft_consensus.cc:482] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:17:14.138367 15421 raft_consensus.cc:504] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:17:14.139437 15421 leader_election.cc:288] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fd81512c7924e789ad904a4e8c4eb29 (127.16.217.67:40165), 54df08210dcd4b84973990752dc3f55f (127.16.217.66:42703)
W20260812 03:17:14.140954 14663 leader_election.cc:341] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 54df08210dcd4b84973990752dc3f55f (127.16.217.66:42703): Not found: Tablet not found: 0d0d301f5a6843589469ab3f5c9cb11e
W20260812 03:17:14.141167 14671 leader_election.cc:341] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fd81512c7924e789ad904a4e8c4eb29 (127.16.217.67:40165): Not found: Tablet not found: 0d0d301f5a6843589469ab3f5c9cb11e
I20260812 03:17:14.141242 14671 leader_election.cc:302] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4f0fa8043f941ae81e2e2ba831c4814; no voters: 0fd81512c7924e789ad904a4e8c4eb29, 54df08210dcd4b84973990752dc3f55f
I20260812 03:17:14.141515 15421 raft_consensus.cc:2726] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:17:14.712882 15423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:14.712423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:55939)
0812 03:17:14.712720 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:14.712783 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":97,"threads_started":1}
I20260812 03:17:14.860265 15425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:14.859833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:54051)
0812 03:17:14.860090 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:14.860161 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":79,"threads_started":1}
I20260812 03:17:15.106124 15426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:15.105655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:48687)
0812 03:17:15.105944 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:15.106020 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":78,"threads_started":1}
I20260812 03:17:15.714327 15427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:15.713789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:33629)
0812 03:17:15.714096 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:15.714150 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":92,"threads_started":1}
I20260812 03:17:15.861491 15429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:15.861010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:48399)
0812 03:17:15.861304 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:15.861391 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":88,"threads_started":1}
I20260812 03:17:16.107429 15432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:16.106942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:56007)
0812 03:17:16.107277 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:16.107336 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":98,"threads_started":1}
I20260812 03:17:16.715531 15434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:16.715114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:40053)
0812 03:17:16.715378 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:16.715434 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":89,"threads_started":1}
I20260812 03:17:16.862938 15436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:16.862483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:48871)
0812 03:17:16.862769 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:16.862838 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":87,"threads_started":1}
I20260812 03:17:17.066012 15437 raft_consensus.cc:482] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:17:17.066195 15437 raft_consensus.cc:504] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:17:17.066715 15437 leader_election.cc:288] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fd81512c7924e789ad904a4e8c4eb29 (127.16.217.67:40165), 54df08210dcd4b84973990752dc3f55f (127.16.217.66:42703)
W20260812 03:17:17.067724 14663 leader_election.cc:341] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 54df08210dcd4b84973990752dc3f55f (127.16.217.66:42703): Not found: Tablet not found: 0d0d301f5a6843589469ab3f5c9cb11e
W20260812 03:17:17.067829 14671 leader_election.cc:341] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fd81512c7924e789ad904a4e8c4eb29 (127.16.217.67:40165): Not found: Tablet not found: 0d0d301f5a6843589469ab3f5c9cb11e
I20260812 03:17:17.067896 14671 leader_election.cc:302] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4f0fa8043f941ae81e2e2ba831c4814; no voters: 0fd81512c7924e789ad904a4e8c4eb29, 54df08210dcd4b84973990752dc3f55f
I20260812 03:17:17.068035 15437 raft_consensus.cc:2726] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:17:17.108745 15439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:17.108282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:46353)
0812 03:17:17.108584 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:17.108648 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":96,"threads_started":1}
I20260812 03:17:17.716698 15442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:17.716289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:56389)
0812 03:17:17.716538 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:17.716595 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":76,"threads_started":1}
I20260812 03:17:17.864180 15443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:17.863721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:46671)
0812 03:17:17.864023 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:17.864085 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":92,"threads_started":1}
I20260812 03:17:18.110050 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:18.109579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:46863)
0812 03:17:18.109891 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:18.109954 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":79,"threads_started":1}
I20260812 03:17:18.718031 15447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:18.717569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:36235)
0812 03:17:18.717859 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:18.717915 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":89,"threads_started":1}
I20260812 03:17:18.865419 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:18.864950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:48149)
0812 03:17:18.865254 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:18.865313 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":113,"threads_started":1}
I20260812 03:17:19.111315 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:19.110861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:36891)
0812 03:17:19.111132 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:19.111198 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":79,"threads_started":1}
I20260812 03:17:19.719457 15451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:19.718917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:60363)
0812 03:17:19.719297 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:19.719357 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":177,"threads_started":1}
I20260812 03:17:19.866799 15452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:19.866324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:54061)
0812 03:17:19.866638 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:19.866702 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":101,"threads_started":1}
I20260812 03:17:20.112818 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:20.112319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:57565)
0812 03:17:20.112623 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:20.112697 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":100,"threads_started":1}
I20260812 03:17:20.720870 15456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:20.720426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:44459)
0812 03:17:20.720682 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:20.720745 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":76,"threads_started":1}
I20260812 03:17:20.868085 15459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:20.867625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:38649)
0812 03:17:20.867914 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:20.867975 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":82,"threads_started":1}
I20260812 03:17:21.114146 15460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:21.113703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:37161)
0812 03:17:21.113981 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:21.114038 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":80,"threads_started":1}
I20260812 03:17:21.722631 15463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:21.722036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:53233)
0812 03:17:21.722451 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:21.722513 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":84,"threads_started":1}
I20260812 03:17:21.869725 15464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:21.868997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:34899)
0812 03:17:21.869476 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:21.869552 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":163,"threads_started":1}
I20260812 03:17:22.071069 15466 raft_consensus.cc:482] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:17:22.071224 15466 raft_consensus.cc:504] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd81512c7924e789ad904a4e8c4eb29" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40165 } } peers { permanent_uuid: "54df08210dcd4b84973990752dc3f55f" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42703 } } peers { permanent_uuid: "e4f0fa8043f941ae81e2e2ba831c4814" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45103 } }
I20260812 03:17:22.071755 15466 leader_election.cc:288] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fd81512c7924e789ad904a4e8c4eb29 (127.16.217.67:40165), 54df08210dcd4b84973990752dc3f55f (127.16.217.66:42703)
W20260812 03:17:22.072599 14671 leader_election.cc:341] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fd81512c7924e789ad904a4e8c4eb29 (127.16.217.67:40165): Not found: Tablet not found: 0d0d301f5a6843589469ab3f5c9cb11e
W20260812 03:17:22.072749 14663 leader_election.cc:341] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 54df08210dcd4b84973990752dc3f55f (127.16.217.66:42703): Not found: Tablet not found: 0d0d301f5a6843589469ab3f5c9cb11e
I20260812 03:17:22.072852 14663 leader_election.cc:302] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4f0fa8043f941ae81e2e2ba831c4814; no voters: 0fd81512c7924e789ad904a4e8c4eb29, 54df08210dcd4b84973990752dc3f55f
I20260812 03:17:22.073051 15466 raft_consensus.cc:2726] T 0d0d301f5a6843589469ab3f5c9cb11e P e4f0fa8043f941ae81e2e2ba831c4814 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:17:22.115414 15467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:22.114934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:57111)
0812 03:17:22.115252 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:22.115318 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":167,"threads_started":1}
I20260812 03:17:22.724108 15468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:22.723639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:53831)
0812 03:17:22.723914 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:22.723975 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":80,"threads_started":1}
I20260812 03:17:22.871073 15469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:22.870600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:35993)
0812 03:17:22.870889 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:22.870952 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":84,"threads_started":1}
I20260812 03:17:23.116799 15471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:23.116308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:55979)
0812 03:17:23.116624 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:23.116690 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":82,"threads_started":1}
I20260812 03:17:23.725576 15473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:23.725138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:41885)
0812 03:17:23.725401 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:23.725472 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":78,"threads_started":1}
I20260812 03:17:23.872479 15474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:23.871974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:55451)
0812 03:17:23.872308 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:23.872374 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":107,"threads_started":1}
I20260812 03:17:24.118211 15477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:24.117669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:60991)
0812 03:17:24.117990 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:24.118096 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":97,"threads_started":1}
I20260812 03:17:24.726843 15478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:24.726404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:58199)
0812 03:17:24.726687 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:24.726745 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":77,"threads_started":1}
I20260812 03:17:24.873873 15479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:24.873400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:42327)
0812 03:17:24.873713 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:24.873773 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":104,"threads_started":1}
I20260812 03:17:25.119439 15480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:25.119018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:40431)
0812 03:17:25.119290 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:25.119341 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":92,"threads_started":1}
I20260812 03:17:25.728401 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:25.727914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.65:48373)
0812 03:17:25.728222 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:25.728301 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":84,"threads_started":1}
I20260812 03:17:25.875844 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:25.875126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.66:33521)
0812 03:17:25.875579 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:25.875684 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":83,"threads_started":1}
I20260812 03:17:26.106875 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 14562
I20260812 03:17:26.120975 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:26.120535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39545 (local address 127.16.217.67:44051)
0812 03:17:26.120817 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:26.120875 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.126:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":103,"threads_started":1}
I20260812 03:17:26.125380 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 14900
I20260812 03:17:26.136845 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 15133
2026-08-12T03:17:26Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (60022 ms)
[----------] 30 tests from SecurityITest (263168 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:17:26 dist-test-slave-7nm7 krb5kdc[15495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:17:26 dist-test-slave-7nm7 krb5kdc[15495](info): set up 2 sockets
Aug 12 03:17:26 dist-test-slave-7nm7 krb5kdc[15495](info): commencing operation
krb5kdc: starting...
W20260812 03:17:27.489830 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.319s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:17:27 dist-test-slave-7nm7 krb5kdc[15495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:17:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:17:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:17:27.629822 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:45585
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:44279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:45585
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:27.746223 15516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:27.746591 15516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:27.746667 15516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:27.750747 15516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:27.750865 15516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:27.750905 15516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:27.750967 15516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:27.751005 15516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:27.756409 15516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:44279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:45585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:45585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:27.757581 15516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:27.758703 15516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:27.768183 15526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:27.768250 15516 server_base.cc:1034] running on GCE node
W20260812 03:17:27.768177 15532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:27.768203 15528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:27.768956 15516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:27.770087 15516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:27.771292 15516 hybrid_clock.cc:648] HybridClock initialized: now 1786504647771270 us; error 32 us; skew 500 ppm
Aug 12 03:17:27 dist-test-slave-7nm7 krb5kdc[15495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504647, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:27.774487 15516 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:17:27.775434 15516 webserver.cc:460] Webserver started at http://127.16.217.126:35319/ using document root <none> and password file <none>
I20260812 03:17:27.776013 15516 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:27.776074 15516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:27.776320 15516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:27.778501 15516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "dd64bb7c61444aa2b7d2f9cfa0c9a76c"
format_stamp: "Formatted at 2026-08-12 03:17:27 on dist-test-slave-7nm7"
server_key: "27754eb7ccdc8afc18020bdcde853edd"
server_key_iv: "57906f8c1b4f27857e52d329e92d9f70"
server_key_version: "encryptionkey@0"
I20260812 03:17:27.779101 15516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "dd64bb7c61444aa2b7d2f9cfa0c9a76c"
format_stamp: "Formatted at 2026-08-12 03:17:27 on dist-test-slave-7nm7"
server_key: "27754eb7ccdc8afc18020bdcde853edd"
server_key_iv: "57906f8c1b4f27857e52d329e92d9f70"
server_key_version: "encryptionkey@0"
I20260812 03:17:27.783120 15516 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:17:27.785776 15546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:27.787089 15516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:17:27.787259 15516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "dd64bb7c61444aa2b7d2f9cfa0c9a76c"
format_stamp: "Formatted at 2026-08-12 03:17:27 on dist-test-slave-7nm7"
server_key: "27754eb7ccdc8afc18020bdcde853edd"
server_key_iv: "57906f8c1b4f27857e52d329e92d9f70"
server_key_version: "encryptionkey@0"
I20260812 03:17:27.787388 15516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:27.817382 15516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:27.821027 15516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:27.821274 15516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:27.829284 15516 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:45585
I20260812 03:17:27.829346 15649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:45585 every 8 connection(s)
I20260812 03:17:27.830327 15516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:17:27.833230 15651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:27.838644 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 15516
I20260812 03:17:27.838796 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:17:27.839115 17253 external_mini_cluster.cc:1428] Setting key 0d5f649de6f6a0d6322821f6f4af14f7
I20260812 03:17:27.839416 15651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c: Bootstrap starting.
I20260812 03:17:27.842125 15651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:27.842940 15651 log.cc:824] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:27.845002 15651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c: No bootstrap required, opened a new log
Aug 12 03:17:27 dist-test-slave-7nm7 krb5kdc[15495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:17:27.848815 15651 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd64bb7c61444aa2b7d2f9cfa0c9a76c" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 45585 } }
I20260812 03:17:27.849148 15651 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:27.849231 15651 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd64bb7c61444aa2b7d2f9cfa0c9a76c, State: Initialized, Role: FOLLOWER
I20260812 03:17:27.849893 15651 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd64bb7c61444aa2b7d2f9cfa0c9a76c" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 45585 } }
I20260812 03:17:27.850111 15651 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:17:27.850193 15651 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:17:27.850337 15651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:27.851603 15651 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd64bb7c61444aa2b7d2f9cfa0c9a76c" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 45585 } }
I20260812 03:17:27.851994 15651 leader_election.cc:302] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd64bb7c61444aa2b7d2f9cfa0c9a76c; no voters:
I20260812 03:17:27.852411 15651 leader_election.cc:288] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:17:27.852548 15659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:27.852823 15659 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c [term 1 LEADER]: Becoming Leader. State: Replica: dd64bb7c61444aa2b7d2f9cfa0c9a76c, State: Running, Role: LEADER
I20260812 03:17:27.853225 15659 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd64bb7c61444aa2b7d2f9cfa0c9a76c" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 45585 } }
I20260812 03:17:27.854064 15651 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:17:27.855237 15664 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd64bb7c61444aa2b7d2f9cfa0c9a76c. Latest consensus state: current_term: 1 leader_uuid: "dd64bb7c61444aa2b7d2f9cfa0c9a76c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd64bb7c61444aa2b7d2f9cfa0c9a76c" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 45585 } } }
I20260812 03:17:27.855301 15656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:27.840715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60158 (local address 127.16.217.126:45585)
0812 03:17:27.841182 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:27.841201 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:27.841238 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:17:27.841889 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:27.841906 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:27.842327 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:27.842672 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:27.843491 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:27.844503 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:27.845471 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:27.845950 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:27.846127 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:27.848750 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:27.848755 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:27.848770 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:27.848799 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:27.851027 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:27.851856 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:27.851864 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:27.851870 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:27.852026 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:27.852537 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:27.852543 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:27.852545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:27.853164 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:27.853412 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:27.853826 (+ 414us) server_negotiation.cc:299] Negotiation successful
0812 03:17:27.854150 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":141,"threads_started":1}
I20260812 03:17:27.855381 15664 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:27.856182 15663 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd64bb7c61444aa2b7d2f9cfa0c9a76c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd64bb7c61444aa2b7d2f9cfa0c9a76c" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 45585 } } }
I20260812 03:17:27.856287 15663 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:27.856921 15680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:17:27.860695 15680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:17:27.867451 15680 catalog_manager.cc:1347] Generated new cluster ID: 43e09625ace6458789275a75d9945972
I20260812 03:17:27.867542 15680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:17:27.886688 15680 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:17:27.887758 15680 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:17:27.894511 15680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c: Generated new TSK 0
I20260812 03:17:27.895293 15680 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
I20260812 03:17:27.934387 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:45585
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:44279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:28.074486 15706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:28.074817 15706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:28.074900 15706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:28.079337 15706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:28.079411 15706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:28.079473 15706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:17:28.085207 15706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:44279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:45585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:28.086267 15706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:28.087260 15706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:28.097203 15720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:28.097213 15723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:28.097178 15715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:28.097467 15706 server_base.cc:1034] running on GCE node
I20260812 03:17:28.097921 15706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:28.098651 15706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:28.099858 15706 hybrid_clock.cc:648] HybridClock initialized: now 1786504648099824 us; error 47 us; skew 500 ppm
Aug 12 03:17:28 dist-test-slave-7nm7 krb5kdc[15495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504648, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:28.102921 15706 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:17:28.103875 15706 webserver.cc:460] Webserver started at http://127.16.217.65:44957/ using document root <none> and password file <none>
I20260812 03:17:28.104475 15706 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:28.104570 15706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:28.104832 15706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:28.107697 15706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "0f29873c862c4684b9a27a3bfdce2295"
format_stamp: "Formatted at 2026-08-12 03:17:28 on dist-test-slave-7nm7"
server_key: "eaa60efa5c5f213b92f1d2cad98d6b29"
server_key_iv: "dee48927ec39da1e340f08a467460ffc"
server_key_version: "encryptionkey@0"
I20260812 03:17:28.108355 15706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "0f29873c862c4684b9a27a3bfdce2295"
format_stamp: "Formatted at 2026-08-12 03:17:28 on dist-test-slave-7nm7"
server_key: "eaa60efa5c5f213b92f1d2cad98d6b29"
server_key_iv: "dee48927ec39da1e340f08a467460ffc"
server_key_version: "encryptionkey@0"
I20260812 03:17:28.112564 15706 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:17:28.115285 15734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:28.116606 15706 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:17:28.116762 15706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "0f29873c862c4684b9a27a3bfdce2295"
format_stamp: "Formatted at 2026-08-12 03:17:28 on dist-test-slave-7nm7"
server_key: "eaa60efa5c5f213b92f1d2cad98d6b29"
server_key_iv: "dee48927ec39da1e340f08a467460ffc"
server_key_version: "encryptionkey@0"
I20260812 03:17:28.116885 15706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:28.130512 15706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:28.133756 15706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:28.133977 15706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:28.134683 15706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:28.135562 15706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:28.135617 15706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:28.135686 15706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:28.135717 15706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:28.144940 15706 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:42487
I20260812 03:17:28.145013 15934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:42487 every 8 connection(s)
I20260812 03:17:28.145861 15706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:17:28.146327 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 15706
I20260812 03:17:28.146487 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:17:28.146912 17253 external_mini_cluster.cc:1428] Setting key c08c24d076750b11b8dbf8e0f3a74103
Aug 12 03:17:28 dist-test-slave-7nm7 krb5kdc[15495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504648, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.217.66@KRBTEST.COM; defaulting to no policy
I20260812 03:17:28.160858 15656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:28.148543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:43229 (local address 127.16.217.126:45585)
0812 03:17:28.148690 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:28.148695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:28.149297 (+ 602us) server_negotiation.cc:407] Connection header received
0812 03:17:28.150742 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:28.150745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:28.150804 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:28.150908 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:28.152053 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.152582 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.153724 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.154007 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.154125 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:28.157100 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:28.157105 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:28.157109 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:28.157144 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:28.158839 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:28.159408 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:28.159411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:28.159412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:28.159488 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:28.159847 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:28.159850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:28.159851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:28.160080 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:28.160162 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:28.160624 (+ 462us) server_negotiation.cc:299] Negotiation successful
0812 03:17:28.160734 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.66@KRBTEST.COM" created.
I20260812 03:17:28.161892 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:28.148551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:45585 (local address 127.16.217.65:43229)
0812 03:17:28.149127 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:28.149169 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:28.150529 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:28.151077 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:28.151085 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:28.151432 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:28.151906 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:28.151918 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.152715 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.152719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:28.153590 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:28.153599 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.154355 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.154362 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:28.154599 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:28.155374 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:17:28.155391 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:28.156913 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:28.158973 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:28.158978 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:28.158990 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:28.159308 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:28.159609 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:28.159612 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:28.159613 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:28.159754 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:28.160177 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:28.160182 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:28.160414 (+ 232us) client_negotiation.cc:769] Sending connection context
0812 03:17:28.160573 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 03:17:28.160822 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":189,"threads_started":1}
I20260812 03:17:28.163681 15937 heartbeater.cc:347] Connected to a master server at 127.16.217.126:45585
I20260812 03:17:28.163988 15937 heartbeater.cc:464] Registering TS with master...
I20260812 03:17:28.164623 15937 heartbeater.cc:511] Master 127.16.217.126:45585 requested a full tablet report, sending...
I20260812 03:17:28.166532 15589 ts_manager.cc:194] Registered new tserver with Master: 0f29873c862c4684b9a27a3bfdce2295 (127.16.217.65:42487)
I20260812 03:17:28.168265 15589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:43229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
I20260812 03:17:28.179729 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:45585
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:44279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:28.299326 15950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:28.299644 15950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:28.299741 15950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:28.304172 15950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:28.304270 15950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:28.304359 15950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:17:28.309991 15950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:44279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:45585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:28.311203 15950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:28.312294 15950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:28.323532 15964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:28.323529 15960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:28.323601 15959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:28.323886 15950 server_base.cc:1034] running on GCE node
I20260812 03:17:28.324543 15950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:28.325222 15950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:28.326426 15950 hybrid_clock.cc:648] HybridClock initialized: now 1786504648326390 us; error 51 us; skew 500 ppm
Aug 12 03:17:28 dist-test-slave-7nm7 krb5kdc[15495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504648, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:28.329341 15950 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:17:28.330368 15950 webserver.cc:460] Webserver started at http://127.16.217.66:34979/ using document root <none> and password file <none>
I20260812 03:17:28.330945 15950 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:28.331008 15950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:28.331192 15950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:28.333297 15950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "2eeeb90a5ce246c1bd9c79363c69f79a"
format_stamp: "Formatted at 2026-08-12 03:17:28 on dist-test-slave-7nm7"
server_key: "3525d09be79bb5517f8548b131501f8d"
server_key_iv: "a19e89d68153aa051a0b04ffe894704a"
server_key_version: "encryptionkey@0"
I20260812 03:17:28.333833 15950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "2eeeb90a5ce246c1bd9c79363c69f79a"
format_stamp: "Formatted at 2026-08-12 03:17:28 on dist-test-slave-7nm7"
server_key: "3525d09be79bb5517f8548b131501f8d"
server_key_iv: "a19e89d68153aa051a0b04ffe894704a"
server_key_version: "encryptionkey@0"
I20260812 03:17:28.337850 15950 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 03:17:28.341313 15972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:28.343000 15950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 03:17:28.343147 15950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "2eeeb90a5ce246c1bd9c79363c69f79a"
format_stamp: "Formatted at 2026-08-12 03:17:28 on dist-test-slave-7nm7"
server_key: "3525d09be79bb5517f8548b131501f8d"
server_key_iv: "a19e89d68153aa051a0b04ffe894704a"
server_key_version: "encryptionkey@0"
I20260812 03:17:28.343333 15950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:28.367183 15950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:28.370852 15950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:28.371022 15950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:28.371608 15950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:28.372470 15950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:28.372520 15950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:28.372560 15950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:28.372584 15950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:28.384829 15950 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:45205
I20260812 03:17:28.384853 16176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:45205 every 8 connection(s)
I20260812 03:17:28.385807 15950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:17:28.388006 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 15950
I20260812 03:17:28.388139 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:17:28.388459 17253 external_mini_cluster.cc:1428] Setting key 1f0ffab1cdb19f7b55af629b1b7a35a7
Aug 12 03:17:28 dist-test-slave-7nm7 krb5kdc[15495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504648, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.217.67@KRBTEST.COM; defaulting to no policy
I20260812 03:17:28.401511 15656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:28.387808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:45889 (local address 127.16.217.126:45585)
0812 03:17:28.387997 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:28.388002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:28.388889 (+ 887us) server_negotiation.cc:407] Connection header received
0812 03:17:28.390504 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:28.390508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:28.390566 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:28.390645 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:28.391895 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.392462 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.393871 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.394158 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.394343 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:28.397333 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:28.397337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:28.397339 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:28.397365 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:28.398991 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:28.399884 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:28.399888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:28.399889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:28.399987 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:28.400388 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:28.400391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:28.400392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:28.400597 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:28.400682 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:28.401258 (+ 576us) server_negotiation.cc:299] Negotiation successful
0812 03:17:28.401381 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.67@KRBTEST.COM" created.
I20260812 03:17:28.402608 16186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:28.388236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:45585 (local address 127.16.217.66:45889)
0812 03:17:28.388735 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:28.388790 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:28.390223 (+ 1433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:28.390822 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:28.390829 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:28.391239 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:28.391745 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:28.391759 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.392615 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.392618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:28.393736 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:28.393747 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.394384 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.394389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:28.394566 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:28.395306 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:17:28.395324 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:28.397084 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:28.399323 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:28.399330 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:28.399350 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:28.399762 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:28.400119 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:28.400122 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:28.400123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:28.400281 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:28.400727 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:28.400741 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:28.401018 (+ 277us) client_negotiation.cc:769] Sending connection context
0812 03:17:28.401196 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 03:17:28.401476 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":130,"threads_started":1}
I20260812 03:17:28.403946 16183 heartbeater.cc:347] Connected to a master server at 127.16.217.126:45585
I20260812 03:17:28.404225 16183 heartbeater.cc:464] Registering TS with master...
I20260812 03:17:28.404786 16183 heartbeater.cc:511] Master 127.16.217.126:45585 requested a full tablet report, sending...
I20260812 03:17:28.405942 15589 ts_manager.cc:194] Registered new tserver with Master: 2eeeb90a5ce246c1bd9c79363c69f79a (127.16.217.66:45205)
I20260812 03:17:28.406631 15589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:45889
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
I20260812 03:17:28.422472 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:45585
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:44279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:28.551002 16191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:28.551281 16191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:28.551326 16191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:28.555546 16191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:28.555608 16191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:28.555672 16191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:17:28.562178 16191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:44279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:45585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:28.563448 16191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:28.564538 16191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:28.574359 16202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:28.574565 16201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:28.574793 16191 server_base.cc:1034] running on GCE node
W20260812 03:17:28.574867 16204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:28.575313 16191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:28.575970 16191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:28.577163 16191 hybrid_clock.cc:648] HybridClock initialized: now 1786504648577145 us; error 32 us; skew 500 ppm
Aug 12 03:17:28 dist-test-slave-7nm7 krb5kdc[15495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504648, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:28.579994 16191 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:17:28.580888 16191 webserver.cc:460] Webserver started at http://127.16.217.67:39615/ using document root <none> and password file <none>
I20260812 03:17:28.581468 16191 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:28.581568 16191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:28.581818 16191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:28.584014 16191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "48e9638b23da46ca9ab2f0997833374f"
format_stamp: "Formatted at 2026-08-12 03:17:28 on dist-test-slave-7nm7"
server_key: "5457b85d4830d80fe6923047607fd705"
server_key_iv: "79f60924497de85c4e3b3ffc2c32a815"
server_key_version: "encryptionkey@0"
I20260812 03:17:28.584597 16191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "48e9638b23da46ca9ab2f0997833374f"
format_stamp: "Formatted at 2026-08-12 03:17:28 on dist-test-slave-7nm7"
server_key: "5457b85d4830d80fe6923047607fd705"
server_key_iv: "79f60924497de85c4e3b3ffc2c32a815"
server_key_version: "encryptionkey@0"
I20260812 03:17:28.588456 16191 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260812 03:17:28.591172 16214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:28.592597 16191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:17:28.592749 16191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "48e9638b23da46ca9ab2f0997833374f"
format_stamp: "Formatted at 2026-08-12 03:17:28 on dist-test-slave-7nm7"
server_key: "5457b85d4830d80fe6923047607fd705"
server_key_iv: "79f60924497de85c4e3b3ffc2c32a815"
server_key_version: "encryptionkey@0"
I20260812 03:17:28.592886 16191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:28.625244 16191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:28.628815 16191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:28.629055 16191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:28.629733 16191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:28.630720 16191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:28.630792 16191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:28.630878 16191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:28.630927 16191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:28.640499 16191 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:33073
I20260812 03:17:28.640529 16417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:33073 every 8 connection(s)
I20260812 03:17:28.641448 16191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:17:28.651831 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 16191
I20260812 03:17:28.651983 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:17:28.652352 17253 external_mini_cluster.cc:1428] Setting key 7e7d9277621af225ccb81a6d4a55fd2f
Aug 12 03:17:28 dist-test-slave-7nm7 krb5kdc[15495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504648, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:17:28.657068 15656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:28.643683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:54079 (local address 127.16.217.126:45585)
0812 03:17:28.643921 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:28.643927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:28.644573 (+ 646us) server_negotiation.cc:407] Connection header received
0812 03:17:28.645888 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:28.645891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:28.645940 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:28.646017 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:28.647370 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.647907 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.649779 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.650008 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.650151 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:28.653051 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:28.653055 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:28.653057 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:28.653129 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:28.654987 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:28.655612 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:28.655615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:28.655616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:28.655695 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:28.656006 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:28.656008 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:28.656009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:28.656265 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:28.656354 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:28.656845 (+ 491us) server_negotiation.cc:299] Negotiation successful
0812 03:17:28.656944 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260812 03:17:28.657963 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:28.643862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:45585 (local address 127.16.217.67:54079)
0812 03:17:28.644413 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:28.644456 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:28.645687 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:28.646176 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:28.646182 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:28.646633 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:28.647203 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:28.647216 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.648303 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.648309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:28.649628 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:28.649643 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.650201 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.650206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:28.650422 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:28.651035 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:17:28.651055 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:28.652871 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:28.655098 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:28.655157 (+ 59us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:28.655174 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:28.655512 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:28.655783 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:28.655785 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:28.655786 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:28.655922 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:28.656380 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:28.656385 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:28.656630 (+ 245us) client_negotiation.cc:769] Sending connection context
0812 03:17:28.656797 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 03:17:28.657071 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":127,"threads_started":1}
I20260812 03:17:28.659772 16418 heartbeater.cc:347] Connected to a master server at 127.16.217.126:45585
I20260812 03:17:28.660070 16418 heartbeater.cc:464] Registering TS with master...
I20260812 03:17:28.660645 16418 heartbeater.cc:511] Master 127.16.217.126:45585 requested a full tablet report, sending...
I20260812 03:17:28.661756 15589 ts_manager.cc:194] Registered new tserver with Master: 48e9638b23da46ca9ab2f0997833374f (127.16.217.67:33073)
I20260812 03:17:28.662426 15589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:54079
I20260812 03:17:28.666723 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:17:28.676743 15656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:28.668714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60162 (local address 127.16.217.126:45585)
0812 03:17:28.668854 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:28.668859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:28.669148 (+ 289us) server_negotiation.cc:407] Connection header received
0812 03:17:28.669294 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:28.669297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:28.669360 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:28.669459 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:28.670196 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.670780 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.671631 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.671838 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.671925 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:28.673293 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:28.673297 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:28.673299 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:28.673321 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:28.674923 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:28.675412 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:28.675415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:28.675416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:28.675493 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:28.675780 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:28.675783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:28.675784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:28.675990 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:28.676097 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:28.676411 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 03:17:28.676567 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:17:28.679963 15589 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:17:28.683043 15589 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:17:28.697983 16453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:28.693065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34660 (local address 127.16.217.65:42487)
0812 03:17:28.693360 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:28.693388 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:28.693413 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:17:28.693483 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:28.693488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:28.693672 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:28.693824 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:28.694064 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.694955 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.696587 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.697240 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.697360 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:28.697544 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:28.697709 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 03:17:28.697813 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":124,"threads_started":1}
I20260812 03:17:28.698040 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:28.692413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:42487 (local address 127.0.0.1:34660)
0812 03:17:28.693017 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:28.693030 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:28.693115 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:28.693808 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:28.693811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:28.693842 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:28.693939 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:28.693945 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.695043 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.695049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:28.696475 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:28.696486 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.697373 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.697377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:28.697497 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:28.697536 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 03:17:28.697652 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 03:17:28.697786 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":3,"spinlock_wait_cycles":6528,"thread_start_us":93,"threads_started":1}
I20260812 03:17:28.698822 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:28.693419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44454 (local address 127.16.217.67:33073)
0812 03:17:28.694352 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:28.694364 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:28.694385 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:17:28.694455 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:28.694460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:28.694625 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:28.694766 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:28.695002 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.696020 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.697144 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.697923 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.698038 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:28.698209 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:28.698598 (+ 389us) server_negotiation.cc:299] Negotiation successful
0812 03:17:28.698692 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":833,"thread_start_us":111,"threads_started":1}
I20260812 03:17:28.699527 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:28.691931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:45205 (local address 127.0.0.1:54324)
0812 03:17:28.692284 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:28.692512 (+ 228us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:28.692646 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:28.693452 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:28.693458 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:28.693483 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:28.693633 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:28.693640 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.695107 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.695110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:28.697654 (+ 2544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:28.697663 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.699222 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.699228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:28.699307 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:28.699323 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 03:17:28.699380 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 03:17:28.699429 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":1273,"thread_start_us":85,"threads_started":1}
I20260812 03:17:28.699685 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:28.692062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54324 (local address 127.16.217.66:45205)
0812 03:17:28.692919 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:28.692932 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:28.692958 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:17:28.693042 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:28.693047 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:28.693272 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:28.693445 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:28.693775 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.694988 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.697798 (+ 2810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.698877 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.698964 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:28.699104 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:28.699444 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 03:17:28.699540 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":751,"thread_start_us":137,"threads_started":1}
I20260812 03:17:28.701275 15817 tablet_service.cc:1467] Processing CreateTablet for tablet 50c376641dc24d17a28ddfca49e7a907 (DEFAULT_TABLE table=test-table [id=22252d911c444b7d947a7bc5e7753c6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:17:28.702488 15817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50c376641dc24d17a28ddfca49e7a907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:28.702672 16452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:28.692650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:33073 (local address 127.0.0.1:44454)
0812 03:17:28.693736 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:28.693750 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:28.693853 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:28.694770 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:28.694773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:28.694793 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:28.694876 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:28.694881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.696159 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.696166 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:28.697011 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:28.697022 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.698298 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.698302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:28.698402 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:28.698427 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:17:28.702445 (+ 4018us) client_negotiation.cc:240] Negotiation successful
0812 03:17:28.702541 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1014,"mutex_wait_us":1,"spinlock_wait_cycles":768,"thread_start_us":93,"threads_started":1}
I20260812 03:17:28.702332 16059 tablet_service.cc:1467] Processing CreateTablet for tablet 50c376641dc24d17a28ddfca49e7a907 (DEFAULT_TABLE table=test-table [id=22252d911c444b7d947a7bc5e7753c6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:17:28.703281 16059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50c376641dc24d17a28ddfca49e7a907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:28.705122 16302 tablet_service.cc:1467] Processing CreateTablet for tablet 50c376641dc24d17a28ddfca49e7a907 (DEFAULT_TABLE table=test-table [id=22252d911c444b7d947a7bc5e7753c6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:17:28.705905 16302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50c376641dc24d17a28ddfca49e7a907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:28.710321 16456 tablet_bootstrap.cc:492] T 50c376641dc24d17a28ddfca49e7a907 P 0f29873c862c4684b9a27a3bfdce2295: Bootstrap starting.
I20260812 03:17:28.711549 16457 tablet_bootstrap.cc:492] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a: Bootstrap starting.
I20260812 03:17:28.712275 16458 tablet_bootstrap.cc:492] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f: Bootstrap starting.
I20260812 03:17:28.713258 16456 tablet_bootstrap.cc:654] T 50c376641dc24d17a28ddfca49e7a907 P 0f29873c862c4684b9a27a3bfdce2295: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:28.713724 16457 tablet_bootstrap.cc:654] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:28.714198 16456 log.cc:824] T 50c376641dc24d17a28ddfca49e7a907 P 0f29873c862c4684b9a27a3bfdce2295: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:28.714743 16458 tablet_bootstrap.cc:654] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:28.715029 16457 log.cc:824] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:28.715886 16458 log.cc:824] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:28.716481 16456 tablet_bootstrap.cc:492] T 50c376641dc24d17a28ddfca49e7a907 P 0f29873c862c4684b9a27a3bfdce2295: No bootstrap required, opened a new log
I20260812 03:17:28.716697 16456 ts_tablet_manager.cc:1397] T 50c376641dc24d17a28ddfca49e7a907 P 0f29873c862c4684b9a27a3bfdce2295: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 03:17:28.717058 16457 tablet_bootstrap.cc:492] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a: No bootstrap required, opened a new log
I20260812 03:17:28.717245 16457 ts_tablet_manager.cc:1397] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:17:28.717826 16458 tablet_bootstrap.cc:492] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f: No bootstrap required, opened a new log
I20260812 03:17:28.718003 16458 ts_tablet_manager.cc:1397] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:17:28.720777 16456 raft_consensus.cc:348] T 50c376641dc24d17a28ddfca49e7a907 P 0f29873c862c4684b9a27a3bfdce2295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eeeb90a5ce246c1bd9c79363c69f79a" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45205 } } peers { permanent_uuid: "48e9638b23da46ca9ab2f0997833374f" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 33073 } } peers { permanent_uuid: "0f29873c862c4684b9a27a3bfdce2295" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42487 } }
I20260812 03:17:28.721103 16456 raft_consensus.cc:374] T 50c376641dc24d17a28ddfca49e7a907 P 0f29873c862c4684b9a27a3bfdce2295 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:28.721190 16456 raft_consensus.cc:729] T 50c376641dc24d17a28ddfca49e7a907 P 0f29873c862c4684b9a27a3bfdce2295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f29873c862c4684b9a27a3bfdce2295, State: Initialized, Role: FOLLOWER
I20260812 03:17:28.721288 16457 raft_consensus.cc:348] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eeeb90a5ce246c1bd9c79363c69f79a" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45205 } } peers { permanent_uuid: "48e9638b23da46ca9ab2f0997833374f" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 33073 } } peers { permanent_uuid: "0f29873c862c4684b9a27a3bfdce2295" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42487 } }
I20260812 03:17:28.721535 16457 raft_consensus.cc:374] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:28.721606 16457 raft_consensus.cc:729] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eeeb90a5ce246c1bd9c79363c69f79a, State: Initialized, Role: FOLLOWER
I20260812 03:17:28.721686 16456 consensus_queue.cc:260] T 50c376641dc24d17a28ddfca49e7a907 P 0f29873c862c4684b9a27a3bfdce2295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eeeb90a5ce246c1bd9c79363c69f79a" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45205 } } peers { permanent_uuid: "48e9638b23da46ca9ab2f0997833374f" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 33073 } } peers { permanent_uuid: "0f29873c862c4684b9a27a3bfdce2295" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42487 } }
I20260812 03:17:28.721820 16458 raft_consensus.cc:348] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eeeb90a5ce246c1bd9c79363c69f79a" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45205 } } peers { permanent_uuid: "48e9638b23da46ca9ab2f0997833374f" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 33073 } } peers { permanent_uuid: "0f29873c862c4684b9a27a3bfdce2295" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42487 } }
I20260812 03:17:28.722056 16458 raft_consensus.cc:374] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:28.722003 16457 consensus_queue.cc:260] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eeeb90a5ce246c1bd9c79363c69f79a" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45205 } } peers { permanent_uuid: "48e9638b23da46ca9ab2f0997833374f" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 33073 } } peers { permanent_uuid: "0f29873c862c4684b9a27a3bfdce2295" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42487 } }
I20260812 03:17:28.722213 16458 raft_consensus.cc:729] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e9638b23da46ca9ab2f0997833374f, State: Initialized, Role: FOLLOWER
I20260812 03:17:28.722491 15937 heartbeater.cc:503] Master 127.16.217.126:45585 was elected leader, sending a full tablet report...
I20260812 03:17:28.722725 16458 consensus_queue.cc:260] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eeeb90a5ce246c1bd9c79363c69f79a" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45205 } } peers { permanent_uuid: "48e9638b23da46ca9ab2f0997833374f" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 33073 } } peers { permanent_uuid: "0f29873c862c4684b9a27a3bfdce2295" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42487 } }
I20260812 03:17:28.722846 16183 heartbeater.cc:503] Master 127.16.217.126:45585 was elected leader, sending a full tablet report...
I20260812 03:17:28.723059 16457 ts_tablet_manager.cc:1428] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260812 03:17:28.723604 16458 ts_tablet_manager.cc:1428] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260812 03:17:28.723978 16418 heartbeater.cc:503] Master 127.16.217.126:45585 was elected leader, sending a full tablet report...
I20260812 03:17:28.725020 16456 ts_tablet_manager.cc:1428] T 50c376641dc24d17a28ddfca49e7a907 P 0f29873c862c4684b9a27a3bfdce2295: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
W20260812 03:17:28.887147 16185 tablet.cc:2369] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:17:28.892552 16419 tablet.cc:2369] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:17:28.897552 15938 tablet.cc:2369] T 50c376641dc24d17a28ddfca49e7a907 P 0f29873c862c4684b9a27a3bfdce2295: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:17:28.901665 16466 raft_consensus.cc:482] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:17:28.901922 16466 raft_consensus.cc:504] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eeeb90a5ce246c1bd9c79363c69f79a" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45205 } } peers { permanent_uuid: "48e9638b23da46ca9ab2f0997833374f" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 33073 } } peers { permanent_uuid: "0f29873c862c4684b9a27a3bfdce2295" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42487 } }
I20260812 03:17:28.903373 16466 leader_election.cc:288] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2eeeb90a5ce246c1bd9c79363c69f79a (127.16.217.66:45205), 0f29873c862c4684b9a27a3bfdce2295 (127.16.217.65:42487)
I20260812 03:17:28.907531 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:28.903730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:45205 (local address 127.16.217.67:38679)
0812 03:17:28.903874 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:28.903891 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:28.903996 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:28.904479 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:28.904483 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:28.904508 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:28.904667 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:28.904679 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.905559 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.905564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:28.906424 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:28.906433 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.907079 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.907084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:28.907204 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:28.907224 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 03:17:28.907315 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 03:17:28.907387 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 03:17:28.907531 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:28.903852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:38679 (local address 127.16.217.66:45205)
0812 03:17:28.903981 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:28.903986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:28.904004 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:17:28.904216 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:28.904220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:28.904281 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:28.904389 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:28.904817 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.905438 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.906548 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.906947 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.907092 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:28.907155 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:28.907333 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 03:17:28.907403 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:17:28.907887 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:28.903730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:42487 (local address 127.16.217.67:40303)
0812 03:17:28.904306 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:28.904322 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:28.904422 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:28.904701 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:28.904703 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:28.904732 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:28.904881 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:28.904887 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.905828 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.905835 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:28.906699 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:28.906709 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.907416 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.907420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:28.907522 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:28.907539 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 03:17:28.907651 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 03:17:28.907742 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":135,"thread_start_us":205,"threads_started":1}
I20260812 03:17:28.907994 16453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:28.903862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:40303 (local address 127.16.217.65:42487)
0812 03:17:28.903981 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:28.903986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:28.904437 (+ 451us) server_negotiation.cc:407] Connection header received
0812 03:17:28.904507 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:28.904510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:28.904564 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:28.904673 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:28.905007 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.905683 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.906828 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.907280 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.907422 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:28.907473 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:28.907789 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 03:17:28.907862 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:17:28.908269 16101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c376641dc24d17a28ddfca49e7a907" candidate_uuid: "48e9638b23da46ca9ab2f0997833374f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eeeb90a5ce246c1bd9c79363c69f79a" is_pre_election: true
I20260812 03:17:28.908579 15854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c376641dc24d17a28ddfca49e7a907" candidate_uuid: "48e9638b23da46ca9ab2f0997833374f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f29873c862c4684b9a27a3bfdce2295" is_pre_election: true
I20260812 03:17:28.908632 16101 raft_consensus.cc:2454] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48e9638b23da46ca9ab2f0997833374f in term 0.
I20260812 03:17:28.908859 15854 raft_consensus.cc:2454] T 50c376641dc24d17a28ddfca49e7a907 P 0f29873c862c4684b9a27a3bfdce2295 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48e9638b23da46ca9ab2f0997833374f in term 0.
I20260812 03:17:28.909153 16215 leader_election.cc:302] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2eeeb90a5ce246c1bd9c79363c69f79a, 48e9638b23da46ca9ab2f0997833374f; no voters:
I20260812 03:17:28.909473 16466 raft_consensus.cc:2781] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:17:28.909565 16466 raft_consensus.cc:482] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:17:28.909610 16466 raft_consensus.cc:3037] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:28.910769 16466 raft_consensus.cc:504] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eeeb90a5ce246c1bd9c79363c69f79a" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45205 } } peers { permanent_uuid: "48e9638b23da46ca9ab2f0997833374f" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 33073 } } peers { permanent_uuid: "0f29873c862c4684b9a27a3bfdce2295" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42487 } }
I20260812 03:17:28.911154 16466 leader_election.cc:288] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [CANDIDATE]: Term 1 election: Requested vote from peers 2eeeb90a5ce246c1bd9c79363c69f79a (127.16.217.66:45205), 0f29873c862c4684b9a27a3bfdce2295 (127.16.217.65:42487)
I20260812 03:17:28.911509 16101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c376641dc24d17a28ddfca49e7a907" candidate_uuid: "48e9638b23da46ca9ab2f0997833374f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eeeb90a5ce246c1bd9c79363c69f79a"
I20260812 03:17:28.911674 16101 raft_consensus.cc:3037] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:28.911635 15854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c376641dc24d17a28ddfca49e7a907" candidate_uuid: "48e9638b23da46ca9ab2f0997833374f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f29873c862c4684b9a27a3bfdce2295"
I20260812 03:17:28.911837 15854 raft_consensus.cc:3037] T 50c376641dc24d17a28ddfca49e7a907 P 0f29873c862c4684b9a27a3bfdce2295 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:28.912783 16101 raft_consensus.cc:2454] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48e9638b23da46ca9ab2f0997833374f in term 1.
I20260812 03:17:28.912940 15854 raft_consensus.cc:2454] T 50c376641dc24d17a28ddfca49e7a907 P 0f29873c862c4684b9a27a3bfdce2295 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48e9638b23da46ca9ab2f0997833374f in term 1.
I20260812 03:17:28.913163 16215 leader_election.cc:302] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2eeeb90a5ce246c1bd9c79363c69f79a, 48e9638b23da46ca9ab2f0997833374f; no voters:
I20260812 03:17:28.913445 16466 raft_consensus.cc:2781] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:28.913720 16466 raft_consensus.cc:686] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [term 1 LEADER]: Becoming Leader. State: Replica: 48e9638b23da46ca9ab2f0997833374f, State: Running, Role: LEADER
I20260812 03:17:28.914489 16466 consensus_queue.cc:237] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eeeb90a5ce246c1bd9c79363c69f79a" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45205 } } peers { permanent_uuid: "48e9638b23da46ca9ab2f0997833374f" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 33073 } } peers { permanent_uuid: "0f29873c862c4684b9a27a3bfdce2295" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42487 } }
I20260812 03:17:28.917877 15587 catalog_manager.cc:5495] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f reported cstate change: term changed from 0 to 1, leader changed from <none> to 48e9638b23da46ca9ab2f0997833374f (127.16.217.67). New cstate: current_term: 1 leader_uuid: "48e9638b23da46ca9ab2f0997833374f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eeeb90a5ce246c1bd9c79363c69f79a" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48e9638b23da46ca9ab2f0997833374f" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 33073 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f29873c862c4684b9a27a3bfdce2295" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42487 } health_report { overall_health: UNKNOWN } } }
I20260812 03:17:28.933390 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:28.929842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44470 (local address 127.16.217.67:33073)
0812 03:17:28.929975 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:28.929978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:28.929993 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:17:28.930078 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:28.930081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:28.930137 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:28.930278 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:17:28.930507 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.931003 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.931940 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:28.932153 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:28.932233 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:28.932518 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:17:28.932964 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:17:28.933064 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:28.933201 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 03:17:28.933257 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:17:28.939527 16101 raft_consensus.cc:1261] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a [term 1 FOLLOWER]: Refusing update from remote peer 48e9638b23da46ca9ab2f0997833374f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:17:28.939527 15854 raft_consensus.cc:1261] T 50c376641dc24d17a28ddfca49e7a907 P 0f29873c862c4684b9a27a3bfdce2295 [term 1 FOLLOWER]: Refusing update from remote peer 48e9638b23da46ca9ab2f0997833374f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:17:28.940428 16466 consensus_queue.cc:1035] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f29873c862c4684b9a27a3bfdce2295" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:17:28.940629 16468 consensus_queue.cc:1035] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2eeeb90a5ce246c1bd9c79363c69f79a" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:17:28.952033 16480 mvcc.cc:205] Tried to move back new op lower bound from 7317523042048532480 to 7317523041956589568. Current Snapshot: MvccSnapshot[applied={T|T < 7317523042048532480}]
I20260812 03:17:28.953207 16479 mvcc.cc:205] Tried to move back new op lower bound from 7317523042048532480 to 7317523041956589568. Current Snapshot: MvccSnapshot[applied={T|T < 7317523042048532480 or (T in {7317523042048532480})}]
I20260812 03:17:28.959529 15589 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:17:28.959751 15589 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:17:28.962615 15589 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd64bb7c61444aa2b7d2f9cfa0c9a76c: Sending DeleteTablet for 3 replicas of tablet 50c376641dc24d17a28ddfca49e7a907
I20260812 03:17:28.963894 15817 tablet_service.cc:1514] Processing DeleteTablet for tablet 50c376641dc24d17a28ddfca49e7a907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:17:28 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:34660
I20260812 03:17:28.964056 16302 tablet_service.cc:1514] Processing DeleteTablet for tablet 50c376641dc24d17a28ddfca49e7a907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:17:28 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:44454
I20260812 03:17:28.964233 16059 tablet_service.cc:1514] Processing DeleteTablet for tablet 50c376641dc24d17a28ddfca49e7a907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:17:28 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:54324
I20260812 03:17:28.964371 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 15706
I20260812 03:17:28.973148 16497 tablet_replica.cc:330] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f: stopping tablet replica
I20260812 03:17:28.973798 16497 raft_consensus.cc:2229] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:17:28.974501 16497 raft_consensus.cc:2258] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:17:28.978452 16497 ts_tablet_manager.cc:1905] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:17:28.982622 16497 ts_tablet_manager.cc:1918] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:17:28.982841 16497 log.cc:1196] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/50c376641dc24d17a28ddfca49e7a907
I20260812 03:17:28.983311 16497 ts_tablet_manager.cc:1939] T 50c376641dc24d17a28ddfca49e7a907 P 48e9638b23da46ca9ab2f0997833374f: Deleting consensus metadata
I20260812 03:17:28.985057 15549 catalog_manager.cc:4841] TS 48e9638b23da46ca9ab2f0997833374f (127.16.217.67:33073): tablet 50c376641dc24d17a28ddfca49e7a907 (table test-table [id=22252d911c444b7d947a7bc5e7753c6a]) successfully deleted
I20260812 03:17:28.986557 16498 tablet_replica.cc:330] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a: stopping tablet replica
I20260812 03:17:28.986954 16498 raft_consensus.cc:2229] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:17:28.987319 16498 raft_consensus.cc:2258] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:17:28.989142 16498 ts_tablet_manager.cc:1905] T 50c376641dc24d17a28ddfca49e7a907 P 2eeeb90a5ce246c1bd9c79363c69f79a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:17:28.991451 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 15950
W20260812 03:17:28.991474 15552 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.217.65:42487 (error 108)
I20260812 03:17:29.007470 16452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:29.006432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:42487 (local address 127.0.0.1:34676)
0812 03:17:29.007271 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:29.007359 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:42487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":735}
W20260812 03:17:29.007714 15552 catalog_manager.cc:4584] TS 0f29873c862c4684b9a27a3bfdce2295 (127.16.217.65:42487): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 50c376641dc24d17a28ddfca49e7a907: Network error: Client connection negotiation failed: client connection to 127.16.217.65:42487: connect: Connection refused (error 111)
I20260812 03:17:29.013715 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 16191
I20260812 03:17:29.019558 16452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:29.013963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:45205 (local address 127.0.0.1:54336)
0812 03:17:29.018741 (+ 4778us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:29.019435 (+ 694us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.66:45205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4660}
I20260812 03:17:29.038437 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 15516
2026-08-12T03:17:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2900 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T03:17:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:17:29Z Disabled control of system clock
I20260812 03:17:29.082062 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:44957
--webserver_interface=127.16.217.126
--webserver_port=0
--builtin_ntp_servers=127.16.217.84:43071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:44957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:17:29.205662 16505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:29.205945 16505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:29.205983 16505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:29.210172 16505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:29.210265 16505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:29.210282 16505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:29.210300 16505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:29.210316 16505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:29.215173 16505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:44957
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:44957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:29.216475 16505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:29.217923 16505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:29.227286 16518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:29.227286 16515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:29.227346 16513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:29.227851 16505 server_base.cc:1034] running on GCE node
I20260812 03:17:29.228492 16505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:29.229734 16505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:29.230955 16505 hybrid_clock.cc:648] HybridClock initialized: now 1786504649230936 us; error 35 us; skew 500 ppm
I20260812 03:17:29.234107 16505 webserver.cc:460] Webserver started at http://127.16.217.126:38475/ using document root <none> and password file <none>
I20260812 03:17:29.234817 16505 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:29.234918 16505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:29.235176 16505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:29.237401 16505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "5d898a70608f4a028b54d14c75748e86"
format_stamp: "Formatted at 2026-08-12 03:17:29 on dist-test-slave-7nm7"
server_key: "4702c3887c7700c843e4eedbd4581906"
server_key_iv: "538ff239eb3c755ecfc8297c383b72dd"
server_key_version: "encryptionkey@0"
I20260812 03:17:29.237999 16505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "5d898a70608f4a028b54d14c75748e86"
format_stamp: "Formatted at 2026-08-12 03:17:29 on dist-test-slave-7nm7"
server_key: "4702c3887c7700c843e4eedbd4581906"
server_key_iv: "538ff239eb3c755ecfc8297c383b72dd"
server_key_version: "encryptionkey@0"
I20260812 03:17:29.242013 16505 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 03:17:29.244656 16530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:29.246870 16505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:17:29.247062 16505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "5d898a70608f4a028b54d14c75748e86"
format_stamp: "Formatted at 2026-08-12 03:17:29 on dist-test-slave-7nm7"
server_key: "4702c3887c7700c843e4eedbd4581906"
server_key_iv: "538ff239eb3c755ecfc8297c383b72dd"
server_key_version: "encryptionkey@0"
I20260812 03:17:29.247206 16505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:29.261657 16505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:29.262557 16505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:29.262769 16505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:29.270877 16505 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:44957
I20260812 03:17:29.270887 16626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:44957 every 8 connection(s)
I20260812 03:17:29.271914 16505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:17:29.275520 16627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:29.279572 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 16505
I20260812 03:17:29.279742 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:17:29.280062 17253 external_mini_cluster.cc:1428] Setting key 6d28e9a2565d2ae269cec4f1fe72332c
I20260812 03:17:29.282302 16627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86: Bootstrap starting.
I20260812 03:17:29.284904 16627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:29.285734 16627 log.cc:824] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:29.287850 16627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86: No bootstrap required, opened a new log
I20260812 03:17:29.290830 16633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:29.281849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48394 (local address 127.16.217.126:44957)
0812 03:17:29.282616 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:29.282635 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:29.282674 (+ 39us) server_negotiation.cc:407] Connection header received
0812 03:17:29.283532 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:29.283555 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:29.284011 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:29.284316 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:29.284690 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:29.285765 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:29.286795 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:29.287324 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:29.287515 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:29.288436 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:29.288447 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:29.288461 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:29.288868 (+ 407us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:29.288927 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:29.288950 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:29.289186 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:29.289512 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 03:17:29.289795 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":160,"threads_started":1}
I20260812 03:17:29.291785 16627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d898a70608f4a028b54d14c75748e86" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44957 } }
I20260812 03:17:29.292009 16627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:29.292088 16627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d898a70608f4a028b54d14c75748e86, State: Initialized, Role: FOLLOWER
I20260812 03:17:29.292522 16627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d898a70608f4a028b54d14c75748e86" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44957 } }
I20260812 03:17:29.292680 16627 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:17:29.292752 16627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:17:29.292888 16627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:29.293985 16627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d898a70608f4a028b54d14c75748e86" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44957 } }
I20260812 03:17:29.294404 16627 leader_election.cc:302] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d898a70608f4a028b54d14c75748e86; no voters:
I20260812 03:17:29.294757 16627 leader_election.cc:288] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:17:29.294891 16638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:29.295261 16638 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86 [term 1 LEADER]: Becoming Leader. State: Replica: 5d898a70608f4a028b54d14c75748e86, State: Running, Role: LEADER
I20260812 03:17:29.295593 16638 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d898a70608f4a028b54d14c75748e86" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44957 } }
I20260812 03:17:29.296133 16627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:17:29.297287 16640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d898a70608f4a028b54d14c75748e86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d898a70608f4a028b54d14c75748e86" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44957 } } }
I20260812 03:17:29.297426 16640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:29.297791 16644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d898a70608f4a028b54d14c75748e86. Latest consensus state: current_term: 1 leader_uuid: "5d898a70608f4a028b54d14c75748e86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d898a70608f4a028b54d14c75748e86" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44957 } } }
I20260812 03:17:29.297885 16653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:17:29.297964 16644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:29.301528 16653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:17:29.307857 16653 catalog_manager.cc:1347] Generated new cluster ID: cfb5a242dc984077bd51a0778240ce19
I20260812 03:17:29.307947 16653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:17:29.327659 16653 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:17:29.329000 16653 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:17:29.336253 16653 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86: Generated new TSK 0
I20260812 03:17:29.337175 16653 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:17:29.346036 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:44957
--builtin_ntp_servers=127.16.217.84:43071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:17:29.483569 16674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:29.483886 16674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:29.483946 16674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:29.488281 16674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:29.488353 16674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:29.488413 16674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:17:29.495739 16674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:44957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:29.497134 16674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:29.498703 16674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:29.509256 16680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:29.509256 16686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:29.509442 16674 server_base.cc:1034] running on GCE node
W20260812 03:17:29.509531 16681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:29.510100 16674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:29.510835 16674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:29.512144 16674 hybrid_clock.cc:648] HybridClock initialized: now 1786504649512057 us; error 105 us; skew 500 ppm
I20260812 03:17:29.515202 16674 webserver.cc:460] Webserver started at http://127.16.217.65:44163/ using document root <none> and password file <none>
I20260812 03:17:29.515841 16674 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:29.515908 16674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:29.516191 16674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:29.518560 16674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "eeacc3e1331d4e1a8702228c6d86c35e"
format_stamp: "Formatted at 2026-08-12 03:17:29 on dist-test-slave-7nm7"
server_key: "ba73ef567c9dd7c7f4e30b0cd5e62b1e"
server_key_iv: "4b3576048cf8b24d152912580a4c630d"
server_key_version: "encryptionkey@0"
I20260812 03:17:29.519114 16674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "eeacc3e1331d4e1a8702228c6d86c35e"
format_stamp: "Formatted at 2026-08-12 03:17:29 on dist-test-slave-7nm7"
server_key: "ba73ef567c9dd7c7f4e30b0cd5e62b1e"
server_key_iv: "4b3576048cf8b24d152912580a4c630d"
server_key_version: "encryptionkey@0"
I20260812 03:17:29.523162 16674 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:17:29.526225 16700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:29.527822 16674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:17:29.527932 16674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "eeacc3e1331d4e1a8702228c6d86c35e"
format_stamp: "Formatted at 2026-08-12 03:17:29 on dist-test-slave-7nm7"
server_key: "ba73ef567c9dd7c7f4e30b0cd5e62b1e"
server_key_iv: "4b3576048cf8b24d152912580a4c630d"
server_key_version: "encryptionkey@0"
I20260812 03:17:29.528124 16674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:29.581023 16674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:29.581881 16674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:29.582083 16674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:29.582772 16674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:29.583779 16674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:29.583830 16674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:29.583894 16674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:29.583925 16674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:29.593010 16674 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:44139
I20260812 03:17:29.593030 16902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:44139 every 8 connection(s)
I20260812 03:17:29.593988 16674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:17:29.598625 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 16674
I20260812 03:17:29.598783 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:17:29.599115 17253 external_mini_cluster.cc:1428] Setting key 9059c57c56b7fdeddec92126ffcc0134
I20260812 03:17:29.601100 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:44957
--builtin_ntp_servers=127.16.217.84:43071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:17:29.604537 16633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:29.595988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:58975 (local address 127.16.217.126:44957)
0812 03:17:29.596154 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:29.596158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:29.597436 (+ 1278us) server_negotiation.cc:407] Connection header received
0812 03:17:29.599098 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:29.599102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:29.599167 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:29.599260 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:29.600536 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:29.601072 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:29.602268 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:29.602520 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:29.602635 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:29.604040 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:29.604043 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:29.604045 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:29.604058 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:29.604077 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:29.604085 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:29.604162 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:29.604368 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 03:17:29.604423 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 03:17:29.605792 16907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:29.596429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:44957 (local address 127.16.217.65:58975)
0812 03:17:29.597280 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:29.597322 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:29.598825 (+ 1503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:29.599479 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:29.599490 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:29.599614 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:29.600395 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:29.600410 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:29.601211 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:29.601214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:29.602106 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:29.602116 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:29.602706 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:29.602714 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:29.602959 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:29.603700 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:17:29.603704 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:29.603764 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:17:29.603858 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:29.604183 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:29.604195 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:29.604204 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 03:17:29.604352 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 03:17:29.604670 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":109,"threads_started":1}
I20260812 03:17:29.607177 16903 heartbeater.cc:347] Connected to a master server at 127.16.217.126:44957
I20260812 03:17:29.607478 16903 heartbeater.cc:464] Registering TS with master...
I20260812 03:17:29.608063 16903 heartbeater.cc:511] Master 127.16.217.126:44957 requested a full tablet report, sending...
I20260812 03:17:29.609615 16567 ts_manager.cc:194] Registered new tserver with Master: eeacc3e1331d4e1a8702228c6d86c35e (127.16.217.65:44139)
I20260812 03:17:29.610832 16567 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.217.65:58975
W20260812 03:17:29.725870 16909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:29.726222 16909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:29.726363 16909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:29.730486 16909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:29.730581 16909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:29.730676 16909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:17:29.735805 16909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:44957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:29.737089 16909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:29.738613 16909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:29.749023 16920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:29.749023 16925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:29.749102 16922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:29.749135 16909 server_base.cc:1034] running on GCE node
I20260812 03:17:29.749747 16909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:29.750566 16909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:29.751788 16909 hybrid_clock.cc:648] HybridClock initialized: now 1786504649751778 us; error 45 us; skew 500 ppm
I20260812 03:17:29.753831 16909 webserver.cc:460] Webserver started at http://127.16.217.66:41667/ using document root <none> and password file <none>
I20260812 03:17:29.754529 16909 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:29.754628 16909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:29.754886 16909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:29.757067 16909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "eb88dc7fe90348d6ae9108609afc8b97"
format_stamp: "Formatted at 2026-08-12 03:17:29 on dist-test-slave-7nm7"
server_key: "6c4cd74e544c6fdb8997c91fc4e0610e"
server_key_iv: "2f0954c8bed12c9efe7b2324bfdd3e51"
server_key_version: "encryptionkey@0"
I20260812 03:17:29.757697 16909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "eb88dc7fe90348d6ae9108609afc8b97"
format_stamp: "Formatted at 2026-08-12 03:17:29 on dist-test-slave-7nm7"
server_key: "6c4cd74e544c6fdb8997c91fc4e0610e"
server_key_iv: "2f0954c8bed12c9efe7b2324bfdd3e51"
server_key_version: "encryptionkey@0"
I20260812 03:17:29.761854 16909 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:17:29.764649 16934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:29.766754 16909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:17:29.766911 16909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "eb88dc7fe90348d6ae9108609afc8b97"
format_stamp: "Formatted at 2026-08-12 03:17:29 on dist-test-slave-7nm7"
server_key: "6c4cd74e544c6fdb8997c91fc4e0610e"
server_key_iv: "2f0954c8bed12c9efe7b2324bfdd3e51"
server_key_version: "encryptionkey@0"
I20260812 03:17:29.767037 16909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:29.783627 16909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:29.784402 16909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:29.784610 16909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:29.785283 16909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:29.786358 16909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:29.786410 16909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:29.786453 16909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:29.786505 16909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:29.797004 16909 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:45017
I20260812 03:17:29.797021 17132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:45017 every 8 connection(s)
I20260812 03:17:29.798049 16909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:17:29.800449 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 16909
I20260812 03:17:29.800604 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:17:29.800913 17253 external_mini_cluster.cc:1428] Setting key 4666fd647e6645f1a3bde335eeca4b24
I20260812 03:17:29.803592 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:44957
--builtin_ntp_servers=127.16.217.84:43071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:17:29.810316 16633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:29.800161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:56205 (local address 127.16.217.126:44957)
0812 03:17:29.800434 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:29.800442 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:29.801711 (+ 1269us) server_negotiation.cc:407] Connection header received
0812 03:17:29.803296 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:29.803303 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:29.803367 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:29.803474 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:29.804555 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:29.805414 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:29.807486 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:29.807903 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:29.808040 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:29.809636 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:29.809644 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:29.809648 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:29.809668 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:29.809696 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:29.809711 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:29.809799 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:29.809975 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 03:17:29.810058 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 03:17:29.811374 17135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:29.800705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:44957 (local address 127.16.217.66:56205)
0812 03:17:29.801558 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:29.801615 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:29.803066 (+ 1451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:29.803693 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:29.803700 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:29.803848 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:29.804393 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:29.804407 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:29.805602 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:29.805609 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:29.807203 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:29.807214 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:29.808121 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:29.808125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:29.808285 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:29.809006 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:17:29.809008 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:29.809153 (+ 145us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:17:29.809254 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:29.809819 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:29.809828 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:29.809833 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:17:29.809937 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 03:17:29.810190 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":130,"threads_started":1}
I20260812 03:17:29.812786 17133 heartbeater.cc:347] Connected to a master server at 127.16.217.126:44957
I20260812 03:17:29.813043 17133 heartbeater.cc:464] Registering TS with master...
I20260812 03:17:29.813748 17133 heartbeater.cc:511] Master 127.16.217.126:44957 requested a full tablet report, sending...
I20260812 03:17:29.814934 16567 ts_manager.cc:194] Registered new tserver with Master: eb88dc7fe90348d6ae9108609afc8b97 (127.16.217.66:45017)
I20260812 03:17:29.815532 16567 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.217.66:56205
W20260812 03:17:29.930564 17136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:29.930982 17136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:29.931075 17136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:29.935676 17136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:29.935792 17136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:29.935892 17136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:17:29.940990 17136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:44957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:29.942277 17136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:29.943729 17136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:29.953758 17151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:29.953897 17155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:29.953753 17153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:29.953998 17136 server_base.cc:1034] running on GCE node
I20260812 03:17:29.954566 17136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:29.955215 17136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:29.956374 17136 hybrid_clock.cc:648] HybridClock initialized: now 1786504649956356 us; error 29 us; skew 500 ppm
I20260812 03:17:29.958315 17136 webserver.cc:460] Webserver started at http://127.16.217.67:34655/ using document root <none> and password file <none>
I20260812 03:17:29.958900 17136 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:29.958966 17136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:29.959177 17136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:29.961285 17136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "7e190ee6055f4aa2b3237748fc67c775"
format_stamp: "Formatted at 2026-08-12 03:17:29 on dist-test-slave-7nm7"
server_key: "5c572175b0b26b94c65bf2dc919dbdb6"
server_key_iv: "e7735980d8300cf3f08efc2429ff3480"
server_key_version: "encryptionkey@0"
I20260812 03:17:29.961850 17136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "7e190ee6055f4aa2b3237748fc67c775"
format_stamp: "Formatted at 2026-08-12 03:17:29 on dist-test-slave-7nm7"
server_key: "5c572175b0b26b94c65bf2dc919dbdb6"
server_key_iv: "e7735980d8300cf3f08efc2429ff3480"
server_key_version: "encryptionkey@0"
I20260812 03:17:29.965801 17136 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 03:17:29.968526 17171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:29.969805 17136 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:17:29.969957 17136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "7e190ee6055f4aa2b3237748fc67c775"
format_stamp: "Formatted at 2026-08-12 03:17:29 on dist-test-slave-7nm7"
server_key: "5c572175b0b26b94c65bf2dc919dbdb6"
server_key_iv: "e7735980d8300cf3f08efc2429ff3480"
server_key_version: "encryptionkey@0"
I20260812 03:17:29.970105 17136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:29.993934 17136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:29.994735 17136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:29.994943 17136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:29.995570 17136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:29.996663 17136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:29.996737 17136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:29.996811 17136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:29.996865 17136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:30.006301 17136 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:39127
I20260812 03:17:30.006330 17393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:39127 every 8 connection(s)
I20260812 03:17:30.007416 17136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:17:30.013367 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 17136
I20260812 03:17:30.013520 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:17:30.013870 17253 external_mini_cluster.cc:1428] Setting key 767d0b5f9a9841beec71d8f6bbb7979c
I20260812 03:17:30.017725 16633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:30.009574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:52591 (local address 127.16.217.126:44957)
0812 03:17:30.009747 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:30.009751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:30.010874 (+ 1123us) server_negotiation.cc:407] Connection header received
0812 03:17:30.012252 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:30.012255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:30.012309 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:30.012396 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:30.013291 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.014101 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.015596 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.015815 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.015930 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:30.017129 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:30.017132 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:30.017134 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:30.017146 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:30.017163 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:30.017172 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:30.017254 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:30.017510 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 03:17:30.017577 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 03:17:30.018590 17398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:30.009964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:44957 (local address 127.16.217.67:52591)
0812 03:17:30.010720 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:30.010761 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:30.012040 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:30.012555 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:30.012561 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:30.012639 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:30.013116 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:30.013129 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.014296 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.014300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:30.015435 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:30.015445 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.015971 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.015983 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:30.016210 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:30.016802 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:17:30.016803 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:30.016842 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:17:30.016906 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:30.017269 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:30.017280 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:30.017310 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 03:17:30.017486 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 03:17:30.017774 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":141,"threads_started":1}
I20260812 03:17:30.019711 17395 heartbeater.cc:347] Connected to a master server at 127.16.217.126:44957
I20260812 03:17:30.019956 17395 heartbeater.cc:464] Registering TS with master...
I20260812 03:17:30.020500 17395 heartbeater.cc:511] Master 127.16.217.126:44957 requested a full tablet report, sending...
I20260812 03:17:30.021494 16567 ts_manager.cc:194] Registered new tserver with Master: 7e190ee6055f4aa2b3237748fc67c775 (127.16.217.67:39127)
I20260812 03:17:30.022019 16567 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.217.67:52591
I20260812 03:17:30.028331 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:17:30.034438 16633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:30.030412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48398 (local address 127.16.217.126:44957)
0812 03:17:30.030543 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:30.030567 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:30.030814 (+ 247us) server_negotiation.cc:407] Connection header received
0812 03:17:30.030977 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:30.030981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:30.031029 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:30.031126 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:30.031428 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.032003 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.032862 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.033122 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.033229 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:30.033988 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:30.033992 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:30.033994 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:30.034006 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:30.034023 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:30.034030 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:30.034091 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:30.034165 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 03:17:30.034214 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:17:30.037420 16567 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:17:30.040561 16567 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:17:30.056421 17416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:30.050491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33372 (local address 127.16.217.67:39127)
0812 03:17:30.051138 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:30.051148 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:30.051167 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:17:30.051247 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:30.051251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:30.051389 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:30.051943 (+ 554us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:30.052141 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.053280 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.054817 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.055572 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.055692 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:30.055914 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:30.056112 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 03:17:30.056234 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":146,"threads_started":1}
I20260812 03:17:30.056421 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:30.049946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:39127 (local address 127.0.0.1:33372)
0812 03:17:30.050413 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:30.050462 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:30.050604 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:30.051565 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:30.051571 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:30.051606 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:30.051799 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:30.051807 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.053451 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.053453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:30.054669 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:30.054681 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.055714 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.055719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:30.055866 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:30.055917 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 03:17:30.056071 (+ 154us) client_negotiation.cc:240] Negotiation successful
0812 03:17:30.056228 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":71,"spinlock_wait_cycles":76544,"thread_start_us":111,"threads_started":1}
I20260812 03:17:30.056772 17412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:30.049738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:44139 (local address 127.0.0.1:34028)
0812 03:17:30.050412 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:30.050463 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:30.050602 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:30.051564 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:30.051571 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:30.051607 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:30.051939 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:30.051946 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.053435 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.053439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:30.054891 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:30.054901 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.055930 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.055937 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:30.056022 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:30.056072 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 03:17:30.056128 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 03:17:30.056636 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":724,"spinlock_wait_cycles":26240,"thread_start_us":112,"threads_started":1}
I20260812 03:17:30.057086 17420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:30.050879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50916 (local address 127.16.217.66:45017)
0812 03:17:30.051551 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:30.051563 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:30.051587 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:17:30.051665 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:30.051673 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:30.051857 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:30.052125 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:30.052272 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.052994 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.055259 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.055986 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.056526 (+ 540us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:30.056721 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:30.056777 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 03:17:30.056891 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":123,"threads_started":1}
I20260812 03:17:30.057129 17417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:30.050590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:45017 (local address 127.0.0.1:50916)
0812 03:17:30.051153 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:30.051173 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:30.051288 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:30.052055 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:30.052061 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:30.052083 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:30.052179 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:30.052185 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.053120 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.053126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:30.055127 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:30.055135 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.056137 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.056140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:30.056242 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:30.056265 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:17:30.056321 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 03:17:30.056383 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":147,"thread_start_us":120,"threads_started":1}
I20260812 03:17:30.057497 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:30.049902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34028 (local address 127.16.217.65:44139)
0812 03:17:30.051040 (+ 1138us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:30.051047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:30.051075 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:17:30.051157 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:30.051163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:30.051373 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:30.051715 (+ 342us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:30.052065 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.053280 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.055026 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.055790 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.056224 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:30.056405 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:30.056460 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 03:17:30.057073 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":907,"thread_start_us":96,"threads_started":1}
I20260812 03:17:30.059640 17267 tablet_service.cc:1467] Processing CreateTablet for tablet 7e7060dab2534c1599e3b45cf9b95b00 (DEFAULT_TABLE table=test-table [id=abc3ec03967043b693820f4719a78218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:17:30.060137 16792 tablet_service.cc:1467] Processing CreateTablet for tablet 7e7060dab2534c1599e3b45cf9b95b00 (DEFAULT_TABLE table=test-table [id=abc3ec03967043b693820f4719a78218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:17:30.060097 17028 tablet_service.cc:1467] Processing CreateTablet for tablet 7e7060dab2534c1599e3b45cf9b95b00 (DEFAULT_TABLE table=test-table [id=abc3ec03967043b693820f4719a78218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:17:30.060813 17267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e7060dab2534c1599e3b45cf9b95b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:30.061175 16792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e7060dab2534c1599e3b45cf9b95b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:30.061177 17028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e7060dab2534c1599e3b45cf9b95b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:30.068447 17421 tablet_bootstrap.cc:492] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775: Bootstrap starting.
I20260812 03:17:30.069963 17423 tablet_bootstrap.cc:492] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97: Bootstrap starting.
I20260812 03:17:30.070582 17426 tablet_bootstrap.cc:492] T 7e7060dab2534c1599e3b45cf9b95b00 P eeacc3e1331d4e1a8702228c6d86c35e: Bootstrap starting.
I20260812 03:17:30.072676 17426 tablet_bootstrap.cc:654] T 7e7060dab2534c1599e3b45cf9b95b00 P eeacc3e1331d4e1a8702228c6d86c35e: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:30.072824 17421 tablet_bootstrap.cc:654] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:30.072814 17423 tablet_bootstrap.cc:654] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:30.073773 17423 log.cc:824] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:30.073773 17421 log.cc:824] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:30.074358 17426 log.cc:824] T 7e7060dab2534c1599e3b45cf9b95b00 P eeacc3e1331d4e1a8702228c6d86c35e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:30.076228 17423 tablet_bootstrap.cc:492] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97: No bootstrap required, opened a new log
I20260812 03:17:30.076229 17421 tablet_bootstrap.cc:492] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775: No bootstrap required, opened a new log
I20260812 03:17:30.076335 17426 tablet_bootstrap.cc:492] T 7e7060dab2534c1599e3b45cf9b95b00 P eeacc3e1331d4e1a8702228c6d86c35e: No bootstrap required, opened a new log
I20260812 03:17:30.076493 17423 ts_tablet_manager.cc:1397] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 03:17:30.076504 17426 ts_tablet_manager.cc:1397] T 7e7060dab2534c1599e3b45cf9b95b00 P eeacc3e1331d4e1a8702228c6d86c35e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:17:30.076491 17421 ts_tablet_manager.cc:1397] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 03:17:30.080152 17423 raft_consensus.cc:348] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeacc3e1331d4e1a8702228c6d86c35e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 44139 } } peers { permanent_uuid: "7e190ee6055f4aa2b3237748fc67c775" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39127 } } peers { permanent_uuid: "eb88dc7fe90348d6ae9108609afc8b97" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45017 } }
I20260812 03:17:30.080468 17423 raft_consensus.cc:374] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:30.080524 17423 raft_consensus.cc:729] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb88dc7fe90348d6ae9108609afc8b97, State: Initialized, Role: FOLLOWER
I20260812 03:17:30.081030 17423 consensus_queue.cc:260] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeacc3e1331d4e1a8702228c6d86c35e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 44139 } } peers { permanent_uuid: "7e190ee6055f4aa2b3237748fc67c775" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39127 } } peers { permanent_uuid: "eb88dc7fe90348d6ae9108609afc8b97" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45017 } }
I20260812 03:17:30.081790 17133 heartbeater.cc:503] Master 127.16.217.126:44957 was elected leader, sending a full tablet report...
I20260812 03:17:30.082448 17426 raft_consensus.cc:348] T 7e7060dab2534c1599e3b45cf9b95b00 P eeacc3e1331d4e1a8702228c6d86c35e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeacc3e1331d4e1a8702228c6d86c35e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 44139 } } peers { permanent_uuid: "7e190ee6055f4aa2b3237748fc67c775" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39127 } } peers { permanent_uuid: "eb88dc7fe90348d6ae9108609afc8b97" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45017 } }
I20260812 03:17:30.082448 17421 raft_consensus.cc:348] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeacc3e1331d4e1a8702228c6d86c35e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 44139 } } peers { permanent_uuid: "7e190ee6055f4aa2b3237748fc67c775" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39127 } } peers { permanent_uuid: "eb88dc7fe90348d6ae9108609afc8b97" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45017 } }
I20260812 03:17:30.082746 17421 raft_consensus.cc:374] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:30.082746 17426 raft_consensus.cc:374] T 7e7060dab2534c1599e3b45cf9b95b00 P eeacc3e1331d4e1a8702228c6d86c35e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:30.082803 17421 raft_consensus.cc:729] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e190ee6055f4aa2b3237748fc67c775, State: Initialized, Role: FOLLOWER
I20260812 03:17:30.082803 17426 raft_consensus.cc:729] T 7e7060dab2534c1599e3b45cf9b95b00 P eeacc3e1331d4e1a8702228c6d86c35e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eeacc3e1331d4e1a8702228c6d86c35e, State: Initialized, Role: FOLLOWER
I20260812 03:17:30.083729 17426 consensus_queue.cc:260] T 7e7060dab2534c1599e3b45cf9b95b00 P eeacc3e1331d4e1a8702228c6d86c35e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeacc3e1331d4e1a8702228c6d86c35e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 44139 } } peers { permanent_uuid: "7e190ee6055f4aa2b3237748fc67c775" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39127 } } peers { permanent_uuid: "eb88dc7fe90348d6ae9108609afc8b97" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45017 } }
I20260812 03:17:30.083729 17421 consensus_queue.cc:260] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeacc3e1331d4e1a8702228c6d86c35e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 44139 } } peers { permanent_uuid: "7e190ee6055f4aa2b3237748fc67c775" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39127 } } peers { permanent_uuid: "eb88dc7fe90348d6ae9108609afc8b97" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45017 } }
I20260812 03:17:30.084792 17395 heartbeater.cc:503] Master 127.16.217.126:44957 was elected leader, sending a full tablet report...
I20260812 03:17:30.084935 17421 ts_tablet_manager.cc:1428] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775: Time spent starting tablet: real 0.008s user 0.005s sys 0.004s
I20260812 03:17:30.084935 17426 ts_tablet_manager.cc:1428] T 7e7060dab2534c1599e3b45cf9b95b00 P eeacc3e1331d4e1a8702228c6d86c35e: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260812 03:17:30.085557 17423 ts_tablet_manager.cc:1428] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260812 03:17:30.085661 16903 heartbeater.cc:503] Master 127.16.217.126:44957 was elected leader, sending a full tablet report...
W20260812 03:17:30.095340 16905 tablet.cc:2369] T 7e7060dab2534c1599e3b45cf9b95b00 P eeacc3e1331d4e1a8702228c6d86c35e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:17:30.232713 17434 raft_consensus.cc:482] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:17:30.233050 17434 raft_consensus.cc:504] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeacc3e1331d4e1a8702228c6d86c35e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 44139 } } peers { permanent_uuid: "7e190ee6055f4aa2b3237748fc67c775" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39127 } } peers { permanent_uuid: "eb88dc7fe90348d6ae9108609afc8b97" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45017 } }
I20260812 03:17:30.234413 17434 leader_election.cc:288] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eeacc3e1331d4e1a8702228c6d86c35e (127.16.217.65:44139), 7e190ee6055f4aa2b3237748fc67c775 (127.16.217.67:39127)
I20260812 03:17:30.240134 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:30.234706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:57917 (local address 127.16.217.65:44139)
0812 03:17:30.234862 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:30.234866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:30.234978 (+ 112us) server_negotiation.cc:407] Connection header received
0812 03:17:30.235179 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:30.235182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:30.235267 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:30.235376 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:30.235677 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.236800 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.238758 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.239394 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.239519 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:30.239578 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:30.239912 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 03:17:30.239986 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:17:30.240134 17416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:30.234826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:37005 (local address 127.16.217.67:39127)
0812 03:17:30.234989 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:30.234994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:30.235294 (+ 300us) server_negotiation.cc:407] Connection header received
0812 03:17:30.235428 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:30.235431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:30.235482 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:30.235558 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:30.235850 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.236884 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.238758 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.239376 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.239519 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:30.239581 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:30.239912 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 03:17:30.239986 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 03:17:30.240155 17135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:30.234607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:44139 (local address 127.16.217.66:57917)
0812 03:17:30.234878 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:30.234896 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:30.235016 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:30.235396 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:30.235400 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:30.235416 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:30.235524 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:30.235529 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.236988 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.236993 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:30.238586 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:30.238600 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.239531 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.239535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:30.239704 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:30.239791 (+ 87us) client_negotiation.cc:769] Sending connection context
0812 03:17:30.239877 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 03:17:30.239986 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":169}
I20260812 03:17:30.240162 17441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:30.234693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:39127 (local address 127.16.217.66:37005)
0812 03:17:30.235168 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:30.235193 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:30.235302 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:30.235601 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:30.235606 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:30.235628 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:30.235740 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:30.235745 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.237004 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.237006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:30.238586 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:30.238600 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.239531 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.239535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:30.239683 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:30.239796 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 03:17:30.239877 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 03:17:30.239986 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":256,"thread_start_us":152,"threads_started":1}
I20260812 03:17:30.241237 17305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e7060dab2534c1599e3b45cf9b95b00" candidate_uuid: "eb88dc7fe90348d6ae9108609afc8b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e190ee6055f4aa2b3237748fc67c775" is_pre_election: true
I20260812 03:17:30.241230 16819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e7060dab2534c1599e3b45cf9b95b00" candidate_uuid: "eb88dc7fe90348d6ae9108609afc8b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eeacc3e1331d4e1a8702228c6d86c35e" is_pre_election: true
I20260812 03:17:30.241703 16819 raft_consensus.cc:2454] T 7e7060dab2534c1599e3b45cf9b95b00 P eeacc3e1331d4e1a8702228c6d86c35e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb88dc7fe90348d6ae9108609afc8b97 in term 0.
I20260812 03:17:30.241693 17305 raft_consensus.cc:2454] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb88dc7fe90348d6ae9108609afc8b97 in term 0.
I20260812 03:17:30.242527 16936 leader_election.cc:302] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e190ee6055f4aa2b3237748fc67c775, eb88dc7fe90348d6ae9108609afc8b97; no voters:
I20260812 03:17:30.242795 17434 raft_consensus.cc:2781] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:17:30.242933 17434 raft_consensus.cc:482] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:17:30.242982 17434 raft_consensus.cc:3037] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:30.244318 17434 raft_consensus.cc:504] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeacc3e1331d4e1a8702228c6d86c35e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 44139 } } peers { permanent_uuid: "7e190ee6055f4aa2b3237748fc67c775" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39127 } } peers { permanent_uuid: "eb88dc7fe90348d6ae9108609afc8b97" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45017 } }
I20260812 03:17:30.244721 17434 leader_election.cc:288] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [CANDIDATE]: Term 1 election: Requested vote from peers eeacc3e1331d4e1a8702228c6d86c35e (127.16.217.65:44139), 7e190ee6055f4aa2b3237748fc67c775 (127.16.217.67:39127)
I20260812 03:17:30.245288 16819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e7060dab2534c1599e3b45cf9b95b00" candidate_uuid: "eb88dc7fe90348d6ae9108609afc8b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eeacc3e1331d4e1a8702228c6d86c35e"
I20260812 03:17:30.245288 17305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e7060dab2534c1599e3b45cf9b95b00" candidate_uuid: "eb88dc7fe90348d6ae9108609afc8b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e190ee6055f4aa2b3237748fc67c775"
I20260812 03:17:30.245436 16819 raft_consensus.cc:3037] T 7e7060dab2534c1599e3b45cf9b95b00 P eeacc3e1331d4e1a8702228c6d86c35e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:30.245436 17305 raft_consensus.cc:3037] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:30.246877 16819 raft_consensus.cc:2454] T 7e7060dab2534c1599e3b45cf9b95b00 P eeacc3e1331d4e1a8702228c6d86c35e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb88dc7fe90348d6ae9108609afc8b97 in term 1.
I20260812 03:17:30.246877 17305 raft_consensus.cc:2454] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb88dc7fe90348d6ae9108609afc8b97 in term 1.
I20260812 03:17:30.247309 16939 leader_election.cc:302] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb88dc7fe90348d6ae9108609afc8b97, eeacc3e1331d4e1a8702228c6d86c35e; no voters:
I20260812 03:17:30.247548 17434 raft_consensus.cc:2781] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:30.247840 17434 raft_consensus.cc:686] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [term 1 LEADER]: Becoming Leader. State: Replica: eb88dc7fe90348d6ae9108609afc8b97, State: Running, Role: LEADER
I20260812 03:17:30.248557 17434 consensus_queue.cc:237] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeacc3e1331d4e1a8702228c6d86c35e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 44139 } } peers { permanent_uuid: "7e190ee6055f4aa2b3237748fc67c775" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39127 } } peers { permanent_uuid: "eb88dc7fe90348d6ae9108609afc8b97" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45017 } }
I20260812 03:17:30.252066 16566 catalog_manager.cc:5495] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb88dc7fe90348d6ae9108609afc8b97 (127.16.217.66). New cstate: current_term: 1 leader_uuid: "eb88dc7fe90348d6ae9108609afc8b97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeacc3e1331d4e1a8702228c6d86c35e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 44139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e190ee6055f4aa2b3237748fc67c775" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb88dc7fe90348d6ae9108609afc8b97" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 45017 } health_report { overall_health: HEALTHY } } }
W20260812 03:17:30.258523 17396 tablet.cc:2369] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:17:30.299337 17134 tablet.cc:2369] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:17:30.299893 17420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:30.295586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50922 (local address 127.16.217.66:45017)
0812 03:17:30.295741 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:30.295747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:30.295763 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:17:30.295942 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:30.295945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:30.296005 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:30.296204 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:17:30.296461 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.297227 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.298270 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:30.298504 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:30.298619 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:30.298946 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:17:30.299403 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:17:30.299547 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:30.299661 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 03:17:30.299712 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:17:30.305927 16819 raft_consensus.cc:1261] T 7e7060dab2534c1599e3b45cf9b95b00 P eeacc3e1331d4e1a8702228c6d86c35e [term 1 FOLLOWER]: Refusing update from remote peer eb88dc7fe90348d6ae9108609afc8b97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:17:30.305927 17305 raft_consensus.cc:1261] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775 [term 1 FOLLOWER]: Refusing update from remote peer eb88dc7fe90348d6ae9108609afc8b97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:17:30.306766 17434 consensus_queue.cc:1035] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eeacc3e1331d4e1a8702228c6d86c35e" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 44139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:17:30.307017 17444 consensus_queue.cc:1035] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e190ee6055f4aa2b3237748fc67c775" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 39127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:17:30.316387 17454 mvcc.cc:205] Tried to move back new op lower bound from 7317523047645601792 to 7317523047421116416. Current Snapshot: MvccSnapshot[applied={T|T < 7317523047645601792}]
I20260812 03:17:30.318804 17453 mvcc.cc:205] Tried to move back new op lower bound from 7317523047645601792 to 7317523047421116416. Current Snapshot: MvccSnapshot[applied={T|T < 7317523047645601792}]
I20260812 03:17:30.325608 16565 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:17:30.325778 16565 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:17:30.328464 16565 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d898a70608f4a028b54d14c75748e86: Sending DeleteTablet for 3 replicas of tablet 7e7060dab2534c1599e3b45cf9b95b00
I20260812 03:17:30.329452 17267 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e7060dab2534c1599e3b45cf9b95b00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:17:30 UTC) from {username='slave'} at 127.0.0.1:33372
I20260812 03:17:30.329550 17028 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e7060dab2534c1599e3b45cf9b95b00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:17:30 UTC) from {username='slave'} at 127.0.0.1:50916
I20260812 03:17:30.329581 16792 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e7060dab2534c1599e3b45cf9b95b00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:17:30 UTC) from {username='slave'} at 127.0.0.1:34028
I20260812 03:17:30.330017 17471 tablet_replica.cc:330] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775: stopping tablet replica
I20260812 03:17:30.330296 17472 tablet_replica.cc:330] T 7e7060dab2534c1599e3b45cf9b95b00 P eeacc3e1331d4e1a8702228c6d86c35e: stopping tablet replica
I20260812 03:17:30.330387 17471 raft_consensus.cc:2229] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:17:30.330617 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 16674
I20260812 03:17:30.339453 17471 raft_consensus.cc:2258] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:17:30.340207 17468 tablet_replica.cc:330] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97: stopping tablet replica
I20260812 03:17:30.340677 17468 raft_consensus.cc:2229] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:17:30.341251 17468 raft_consensus.cc:2258] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:17:30.344488 17468 ts_tablet_manager.cc:1905] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:17:30.344488 17471 ts_tablet_manager.cc:1905] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:17:30.347410 17471 ts_tablet_manager.cc:1918] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:17:30.347410 17468 ts_tablet_manager.cc:1918] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:17:30.347527 17471 log.cc:1196] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/7e7060dab2534c1599e3b45cf9b95b00
I20260812 03:17:30.347527 17468 log.cc:1196] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/7e7060dab2534c1599e3b45cf9b95b00
I20260812 03:17:30.347880 17471 ts_tablet_manager.cc:1939] T 7e7060dab2534c1599e3b45cf9b95b00 P 7e190ee6055f4aa2b3237748fc67c775: Deleting consensus metadata
I20260812 03:17:30.347884 17468 ts_tablet_manager.cc:1939] T 7e7060dab2534c1599e3b45cf9b95b00 P eb88dc7fe90348d6ae9108609afc8b97: Deleting consensus metadata
I20260812 03:17:30.349264 16543 catalog_manager.cc:4841] TS eb88dc7fe90348d6ae9108609afc8b97 (127.16.217.66:45017): tablet 7e7060dab2534c1599e3b45cf9b95b00 (table test-table [id=abc3ec03967043b693820f4719a78218]) successfully deleted
I20260812 03:17:30.349370 16540 catalog_manager.cc:4841] TS 7e190ee6055f4aa2b3237748fc67c775 (127.16.217.67:39127): tablet 7e7060dab2534c1599e3b45cf9b95b00 (table test-table [id=abc3ec03967043b693820f4719a78218]) successfully deleted
W20260812 03:17:30.355387 16543 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.217.65:44139 (error 108)
I20260812 03:17:30.355834 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 16909
I20260812 03:17:30.379928 17417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:30.379511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:44139 (local address 127.0.0.1:34034)
0812 03:17:30.379698 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:30.379800 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:44139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 03:17:30.380187 16543 catalog_manager.cc:4584] TS eeacc3e1331d4e1a8702228c6d86c35e (127.16.217.65:44139): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7e7060dab2534c1599e3b45cf9b95b00: Network error: Client connection negotiation failed: client connection to 127.16.217.65:44139: connect: Connection refused (error 111)
I20260812 03:17:30.386214 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 17136
I20260812 03:17:30.414939 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 16505
2026-08-12T03:17:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1379 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:17:30 dist-test-slave-7nm7 krb5kdc[17480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:17:30 dist-test-slave-7nm7 krb5kdc[17480](info): set up 2 sockets
Aug 12 03:17:30 dist-test-slave-7nm7 krb5kdc[17480](info): commencing operation
krb5kdc: starting...
W20260812 03:17:31.748318 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:17:31 dist-test-slave-7nm7 krb5kdc[17480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:17:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:17:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:17:31.893579 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:36581
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:34231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:36581
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:32.015326 17505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:32.015604 17505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:32.015645 17505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:32.019781 17505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:32.019840 17505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:32.019860 17505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:32.019876 17505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:32.019892 17505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:32.025336 17505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:34231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:36581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:36581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:32.026515 17505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:32.027575 17505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:32.038028 17517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:32.038021 17514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:32.038056 17513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:32.038247 17505 server_base.cc:1034] running on GCE node
I20260812 03:17:32.038944 17505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:32.040232 17505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:32.041450 17505 hybrid_clock.cc:648] HybridClock initialized: now 1786504652041439 us; error 43 us; skew 500 ppm
Aug 12 03:17:32 dist-test-slave-7nm7 krb5kdc[17480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504652, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:32.047801 17505 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:17:32.048897 17505 webserver.cc:460] Webserver started at http://127.16.217.126:39251/ using document root <none> and password file <none>
I20260812 03:17:32.049537 17505 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:32.049636 17505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:32.049892 17505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:32.052505 17505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "89cb5d96cd7e4a90975fed56f2629920"
format_stamp: "Formatted at 2026-08-12 03:17:32 on dist-test-slave-7nm7"
server_key: "47d8fa47f92786f8fb7881ab4b8f659a"
server_key_iv: "1f247e7a55471969681432c285cb49bb"
server_key_version: "encryptionkey@0"
I20260812 03:17:32.053361 17505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "89cb5d96cd7e4a90975fed56f2629920"
format_stamp: "Formatted at 2026-08-12 03:17:32 on dist-test-slave-7nm7"
server_key: "47d8fa47f92786f8fb7881ab4b8f659a"
server_key_iv: "1f247e7a55471969681432c285cb49bb"
server_key_version: "encryptionkey@0"
I20260812 03:17:32.057374 17505 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:17:32.060376 17528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:32.061621 17505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260812 03:17:32.061786 17505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "89cb5d96cd7e4a90975fed56f2629920"
format_stamp: "Formatted at 2026-08-12 03:17:32 on dist-test-slave-7nm7"
server_key: "47d8fa47f92786f8fb7881ab4b8f659a"
server_key_iv: "1f247e7a55471969681432c285cb49bb"
server_key_version: "encryptionkey@0"
I20260812 03:17:32.061914 17505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:32.084285 17505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:32.087742 17505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:32.087968 17505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:32.095618 17505 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:36581
I20260812 03:17:32.095650 17633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:36581 every 8 connection(s)
I20260812 03:17:32.096608 17505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:17:32.099495 17638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:32.103181 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 17505
I20260812 03:17:32.103322 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:17:32.103636 17253 external_mini_cluster.cc:1428] Setting key 6df2d06dd30dacd2d152ab8161a54fb0
I20260812 03:17:32.106009 17638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920: Bootstrap starting.
I20260812 03:17:32.108290 17638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:32.109088 17638 log.cc:824] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:32.111047 17638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920: No bootstrap required, opened a new log
Aug 12 03:17:32 dist-test-slave-7nm7 krb5kdc[17480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:17:32.115185 17638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89cb5d96cd7e4a90975fed56f2629920" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36581 } }
I20260812 03:17:32.115407 17638 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:32.115464 17638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89cb5d96cd7e4a90975fed56f2629920, State: Initialized, Role: FOLLOWER
I20260812 03:17:32.115962 17638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89cb5d96cd7e4a90975fed56f2629920" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36581 } }
I20260812 03:17:32.116091 17638 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:17:32.116179 17638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:17:32.116293 17638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:32.117471 17638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89cb5d96cd7e4a90975fed56f2629920" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36581 } }
I20260812 03:17:32.117854 17638 leader_election.cc:302] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89cb5d96cd7e4a90975fed56f2629920; no voters:
I20260812 03:17:32.118193 17638 leader_election.cc:288] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:17:32.118376 17644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:32.118633 17644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920 [term 1 LEADER]: Becoming Leader. State: Replica: 89cb5d96cd7e4a90975fed56f2629920, State: Running, Role: LEADER
I20260812 03:17:32.118970 17644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89cb5d96cd7e4a90975fed56f2629920" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36581 } }
I20260812 03:17:32.119217 17641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:32.105597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52108 (local address 127.16.217.126:36581)
0812 03:17:32.106050 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:32.106070 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:32.106138 (+ 68us) server_negotiation.cc:407] Connection header received
0812 03:17:32.106883 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:32.106912 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:32.107258 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:32.107619 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:32.108424 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.109479 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.110399 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.110867 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.111045 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:32.113938 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:32.113944 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:32.113963 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:32.113997 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:32.115823 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:32.116360 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:32.116367 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:32.116373 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:32.116528 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:32.116817 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:32.116820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:32.116822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:32.117372 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:32.117588 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:32.117865 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 03:17:32.118201 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":146,"threads_started":1}
I20260812 03:17:32.119545 17638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:17:32.120828 17645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89cb5d96cd7e4a90975fed56f2629920" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89cb5d96cd7e4a90975fed56f2629920" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36581 } } }
I20260812 03:17:32.120828 17646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89cb5d96cd7e4a90975fed56f2629920. Latest consensus state: current_term: 1 leader_uuid: "89cb5d96cd7e4a90975fed56f2629920" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89cb5d96cd7e4a90975fed56f2629920" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36581 } } }
I20260812 03:17:32.120975 17645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:32.120985 17646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:32.121551 17649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:17:32.125020 17649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:17:32.130934 17649 catalog_manager.cc:1347] Generated new cluster ID: d29b6cfdbf4b4218a939a54bc533142b
I20260812 03:17:32.131018 17649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:17:32.139439 17649 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:17:32.140482 17649 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:17:32.153216 17649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920: Generated new TSK 0
I20260812 03:17:32.154119 17649 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
I20260812 03:17:32.193434 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:36581
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:34231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:32.318931 17683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:32.319306 17683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:32.319422 17683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:32.324513 17683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:32.324604 17683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:32.324720 17683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:17:32.330507 17683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:34231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:36581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:32.331723 17683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:32.332800 17683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:32.343786 17696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:32.343747 17692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:32.343773 17698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:32.343849 17683 server_base.cc:1034] running on GCE node
I20260812 03:17:32.344568 17683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:32.345293 17683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:32.346457 17683 hybrid_clock.cc:648] HybridClock initialized: now 1786504652346415 us; error 56 us; skew 500 ppm
Aug 12 03:17:32 dist-test-slave-7nm7 krb5kdc[17480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504652, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:32.349543 17683 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:17:32.350718 17683 webserver.cc:460] Webserver started at http://127.16.217.65:41843/ using document root <none> and password file <none>
I20260812 03:17:32.351321 17683 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:32.351419 17683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:32.351673 17683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:32.353832 17683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "b217dde889de4e4fa8eaa18bfa9e2f11"
format_stamp: "Formatted at 2026-08-12 03:17:32 on dist-test-slave-7nm7"
server_key: "b8d8499d895b0fe28e768bd4e95fc855"
server_key_iv: "aefdc1705114fb47244bb7f27a66e18b"
server_key_version: "encryptionkey@0"
I20260812 03:17:32.354475 17683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "b217dde889de4e4fa8eaa18bfa9e2f11"
format_stamp: "Formatted at 2026-08-12 03:17:32 on dist-test-slave-7nm7"
server_key: "b8d8499d895b0fe28e768bd4e95fc855"
server_key_iv: "aefdc1705114fb47244bb7f27a66e18b"
server_key_version: "encryptionkey@0"
I20260812 03:17:32.358814 17683 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:17:32.361567 17707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:32.363140 17683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:17:32.363304 17683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "b217dde889de4e4fa8eaa18bfa9e2f11"
format_stamp: "Formatted at 2026-08-12 03:17:32 on dist-test-slave-7nm7"
server_key: "b8d8499d895b0fe28e768bd4e95fc855"
server_key_iv: "aefdc1705114fb47244bb7f27a66e18b"
server_key_version: "encryptionkey@0"
I20260812 03:17:32.363441 17683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:32.392017 17683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:32.395886 17683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:32.396124 17683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:32.396826 17683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:32.397746 17683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:32.397821 17683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:32.397897 17683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:32.397943 17683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:32.410535 17683 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:39207
I20260812 03:17:32.410589 17928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:39207 every 8 connection(s)
I20260812 03:17:32.411531 17683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:17:32.416030 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 17683
I20260812 03:17:32.416334 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:17:32.416795 17253 external_mini_cluster.cc:1428] Setting key 92f263b7a37125c8a45ca1fec375e27f
Aug 12 03:17:32 dist-test-slave-7nm7 krb5kdc[17480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504652, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.217.66@KRBTEST.COM; defaulting to no policy
I20260812 03:17:32.430177 17641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:32.414325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:36851 (local address 127.16.217.126:36581)
0812 03:17:32.414662 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:32.414667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:32.415533 (+ 866us) server_negotiation.cc:407] Connection header received
0812 03:17:32.417066 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:32.417070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:32.417142 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:32.417256 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:32.418702 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.419513 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.421455 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.421852 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.421952 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:32.425729 (+ 3777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:32.425736 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:32.425742 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:32.425788 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:32.427729 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:32.428375 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:32.428382 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:32.428385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:32.428490 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:32.428921 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:32.428925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:32.428927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:32.429218 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:32.429331 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:32.429919 (+ 588us) server_negotiation.cc:299] Negotiation successful
0812 03:17:32.430040 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260812 03:17:32.431182 17933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:32.414708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:36581 (local address 127.16.217.65:36851)
0812 03:17:32.415369 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:32.415432 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:32.416803 (+ 1371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:32.417495 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:32.417502 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:32.417890 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:32.418547 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:32.418565 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.419680 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.419683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:32.421280 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:32.421296 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.422326 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.422334 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:32.422591 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:32.423373 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:17:32.423401 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:32.425514 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:32.427891 (+ 2377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:32.427897 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:32.427914 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:32.428230 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:32.428621 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:32.428623 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:32.428625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:32.428791 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:32.429366 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:32.429371 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:32.429651 (+ 280us) client_negotiation.cc:769] Sending connection context
0812 03:17:32.429827 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 03:17:32.430124 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":121,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.66@KRBTEST.COM" created.
I20260812 03:17:32.432627 17930 heartbeater.cc:347] Connected to a master server at 127.16.217.126:36581
I20260812 03:17:32.432945 17930 heartbeater.cc:464] Registering TS with master...
I20260812 03:17:32.433605 17930 heartbeater.cc:511] Master 127.16.217.126:36581 requested a full tablet report, sending...
I20260812 03:17:32.435737 17561 ts_manager.cc:194] Registered new tserver with Master: b217dde889de4e4fa8eaa18bfa9e2f11 (127.16.217.65:39207)
I20260812 03:17:32.437235 17561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:36851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
I20260812 03:17:32.451048 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:36581
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:34231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:32.571780 17946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:32.572077 17946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:32.572124 17946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:32.576288 17946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:32.576363 17946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:32.576426 17946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:17:32.581982 17946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:34231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:36581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:32.583295 17946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:32.584383 17946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:32.594937 17951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:32.594909 17952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:32.595150 17956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:32.595336 17946 server_base.cc:1034] running on GCE node
I20260812 03:17:32.595788 17946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:32.596648 17946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:32.597895 17946 hybrid_clock.cc:648] HybridClock initialized: now 1786504652597866 us; error 53 us; skew 500 ppm
Aug 12 03:17:32 dist-test-slave-7nm7 krb5kdc[17480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504652, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:32.600901 17946 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:17:32.602073 17946 webserver.cc:460] Webserver started at http://127.16.217.66:39549/ using document root <none> and password file <none>
I20260812 03:17:32.603431 17946 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:32.603530 17946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:32.603740 17946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:32.606094 17946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "21f79e6e10f44ed893d8308ef924d9a0"
format_stamp: "Formatted at 2026-08-12 03:17:32 on dist-test-slave-7nm7"
server_key: "4cf1847c408de334e01fffb354469385"
server_key_iv: "d4d1b438b33e019ad1c02f6ad7f93441"
server_key_version: "encryptionkey@0"
I20260812 03:17:32.606837 17946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "21f79e6e10f44ed893d8308ef924d9a0"
format_stamp: "Formatted at 2026-08-12 03:17:32 on dist-test-slave-7nm7"
server_key: "4cf1847c408de334e01fffb354469385"
server_key_iv: "d4d1b438b33e019ad1c02f6ad7f93441"
server_key_version: "encryptionkey@0"
I20260812 03:17:32.611285 17946 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:17:32.614151 17972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:32.615597 17946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:17:32.615711 17946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "21f79e6e10f44ed893d8308ef924d9a0"
format_stamp: "Formatted at 2026-08-12 03:17:32 on dist-test-slave-7nm7"
server_key: "4cf1847c408de334e01fffb354469385"
server_key_iv: "d4d1b438b33e019ad1c02f6ad7f93441"
server_key_version: "encryptionkey@0"
I20260812 03:17:32.615808 17946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:32.629741 17946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:32.633476 17946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:32.633642 17946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:32.634224 17946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:32.635161 17946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:32.635210 17946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:32.635251 17946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:32.635267 17946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:32.645044 17946 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:33529
I20260812 03:17:32.645066 18189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:33529 every 8 connection(s)
I20260812 03:17:32.645974 17946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:17:32.650141 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 17946
I20260812 03:17:32.650303 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:17:32.650662 17253 external_mini_cluster.cc:1428] Setting key 66dbae566aa7c91eca35d5997e6cb9af
Aug 12 03:17:32 dist-test-slave-7nm7 krb5kdc[17480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504652, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.67@KRBTEST.COM" created.
I20260812 03:17:32.665597 17641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:32.647899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:53963 (local address 127.16.217.126:36581)
0812 03:17:32.648088 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:32.648093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:32.648969 (+ 876us) server_negotiation.cc:407] Connection header received
0812 03:17:32.650597 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:32.650600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:32.650661 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:32.650749 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:32.652191 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.652993 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.654791 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.655378 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.655590 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:32.659844 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:32.659848 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:32.659850 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:32.659888 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:32.662738 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:32.663445 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:32.663448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:32.663449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:32.663536 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:32.664016 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:32.664018 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:32.664019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:32.664247 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:32.664380 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:32.665120 (+ 740us) server_negotiation.cc:299] Negotiation successful
0812 03:17:32.665307 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 03:17:32.666533 18192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:32.648303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:36581 (local address 127.16.217.66:53963)
0812 03:17:32.648800 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:32.648845 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:32.650321 (+ 1476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:32.650961 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:32.650971 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:32.651375 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:32.652024 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:32.652047 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.653157 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.653160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:32.654649 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:32.654661 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.655694 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.655701 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:32.655964 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:32.657078 (+ 1114us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:17:32.657101 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:32.659487 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:32.662881 (+ 3394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:32.662888 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:32.662907 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:32.663249 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:32.663673 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:32.663679 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:32.663684 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:32.663896 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:32.664382 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:32.664392 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:32.664755 (+ 363us) client_negotiation.cc:769] Sending connection context
0812 03:17:32.664964 (+ 209us) client_negotiation.cc:240] Negotiation successful
0812 03:17:32.665381 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":115,"threads_started":1}
I20260812 03:17:32.667943 18190 heartbeater.cc:347] Connected to a master server at 127.16.217.126:36581
I20260812 03:17:32.668292 18190 heartbeater.cc:464] Registering TS with master...
I20260812 03:17:32.669008 18190 heartbeater.cc:511] Master 127.16.217.126:36581 requested a full tablet report, sending...
I20260812 03:17:32.670185 17561 ts_manager.cc:194] Registered new tserver with Master: 21f79e6e10f44ed893d8308ef924d9a0 (127.16.217.66:33529)
I20260812 03:17:32.670863 17561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:53963
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
I20260812 03:17:32.685894 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:36581
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:34231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:32.812879 18201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:32.813340 18201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:32.813441 18201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:32.817866 18201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:32.817953 18201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:32.818018 18201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:17:32.824678 18201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:34231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:36581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:32.825815 18201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:32.826843 18201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:32.836611 18216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:32.836632 18220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:32.836632 18218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:32.836687 18201 server_base.cc:1034] running on GCE node
I20260812 03:17:32.837293 18201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:32.837968 18201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:32.839169 18201 hybrid_clock.cc:648] HybridClock initialized: now 1786504652839134 us; error 46 us; skew 500 ppm
Aug 12 03:17:32 dist-test-slave-7nm7 krb5kdc[17480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504652, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:32.842133 18201 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:17:32.843199 18201 webserver.cc:460] Webserver started at http://127.16.217.67:34379/ using document root <none> and password file <none>
I20260812 03:17:32.843793 18201 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:32.843888 18201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:32.844197 18201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:32.846334 18201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "3dd235a5f73249319feaca5c870ad46b"
format_stamp: "Formatted at 2026-08-12 03:17:32 on dist-test-slave-7nm7"
server_key: "107285a1be676e14fafbebb772e7fcc0"
server_key_iv: "11c04b1f6563733681e15cf4f2ad5e9c"
server_key_version: "encryptionkey@0"
I20260812 03:17:32.846982 18201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "3dd235a5f73249319feaca5c870ad46b"
format_stamp: "Formatted at 2026-08-12 03:17:32 on dist-test-slave-7nm7"
server_key: "107285a1be676e14fafbebb772e7fcc0"
server_key_iv: "11c04b1f6563733681e15cf4f2ad5e9c"
server_key_version: "encryptionkey@0"
I20260812 03:17:32.850905 18201 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:17:32.853376 18239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:32.854733 18201 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:17:32.854888 18201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "3dd235a5f73249319feaca5c870ad46b"
format_stamp: "Formatted at 2026-08-12 03:17:32 on dist-test-slave-7nm7"
server_key: "107285a1be676e14fafbebb772e7fcc0"
server_key_iv: "11c04b1f6563733681e15cf4f2ad5e9c"
server_key_version: "encryptionkey@0"
I20260812 03:17:32.855008 18201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:32.893848 18201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:32.897321 18201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:32.897565 18201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:32.898248 18201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:32.899195 18201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:32.899276 18201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:32.899356 18201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:32.899406 18201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:32.909024 18201 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:44295
I20260812 03:17:32.909045 18459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:44295 every 8 connection(s)
I20260812 03:17:32.909936 18201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:17:32.915473 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 18201
I20260812 03:17:32.915622 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:17:32.915942 17253 external_mini_cluster.cc:1428] Setting key 3a58af8b944d443ed0d1c19d58cdd6ea
Aug 12 03:17:32 dist-test-slave-7nm7 krb5kdc[17480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504652, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:17:32.925077 17641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:32.911765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:52815 (local address 127.16.217.126:36581)
0812 03:17:32.911920 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:32.911925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:32.912851 (+ 926us) server_negotiation.cc:407] Connection header received
0812 03:17:32.914140 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:32.914144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:32.914191 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:32.914354 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:32.915514 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.916200 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.917640 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.917949 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.918055 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:32.920754 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:32.920758 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:32.920761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:32.920788 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:32.922594 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:32.923322 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:32.923326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:32.923328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:32.923427 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:32.923818 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:32.923823 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:32.923825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:32.924044 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:32.924162 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:32.924744 (+ 582us) server_negotiation.cc:299] Negotiation successful
0812 03:17:32.924877 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:17:32.925910 18465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:32.912208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:36581 (local address 127.16.217.67:52815)
0812 03:17:32.912658 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:32.912731 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:32.913925 (+ 1194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:32.914501 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:32.914514 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:32.914872 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:32.915345 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:32.915358 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.916338 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.916341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:32.917512 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:32.917521 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.918105 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.918109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:32.918299 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:32.918935 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:17:32.918951 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:32.920583 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:32.922730 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:32.922737 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:32.922753 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:32.923202 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:32.923547 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:32.923551 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:32.923553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:32.923710 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:32.924174 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:32.924184 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:32.924460 (+ 276us) client_negotiation.cc:769] Sending connection context
0812 03:17:32.924667 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 03:17:32.925049 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":112,"threads_started":1}
I20260812 03:17:32.927202 18461 heartbeater.cc:347] Connected to a master server at 127.16.217.126:36581
I20260812 03:17:32.927449 18461 heartbeater.cc:464] Registering TS with master...
I20260812 03:17:32.927946 18461 heartbeater.cc:511] Master 127.16.217.126:36581 requested a full tablet report, sending...
I20260812 03:17:32.929083 17561 ts_manager.cc:194] Registered new tserver with Master: 3dd235a5f73249319feaca5c870ad46b (127.16.217.67:44295)
I20260812 03:17:32.929806 17561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:52815
I20260812 03:17:32.929953 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:17:32.942309 17641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:32.932969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52110 (local address 127.16.217.126:36581)
0812 03:17:32.933121 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:32.933127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:32.933883 (+ 756us) server_negotiation.cc:407] Connection header received
0812 03:17:32.934049 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:32.934052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:32.934112 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:32.934211 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:32.935005 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.935692 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.936604 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.936827 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.936934 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:32.938861 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:32.938865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:32.938868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:32.938895 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:32.940357 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:32.940878 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:32.940884 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:32.940891 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:32.940969 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:32.941317 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:32.941321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:32.941323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:32.941531 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:32.941706 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:32.941988 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 03:17:32.942089 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:17:32.945761 17561 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:17:32.948874 17561 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:17:32.964231 18480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:32.957580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56638 (local address 127.16.217.65:39207)
0812 03:17:32.958897 (+ 1317us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:32.958904 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:32.958929 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:17:32.959008 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:32.959013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:32.959221 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:32.959774 (+ 553us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:32.959907 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.961023 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.962837 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.963635 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.963746 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:32.963877 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:32.964000 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 03:17:32.964098 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":832,"thread_start_us":122,"threads_started":1}
I20260812 03:17:32.964215 18478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:32.957421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:39207 (local address 127.0.0.1:56638)
0812 03:17:32.958001 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:32.958054 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:32.958202 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:32.959394 (+ 1192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:32.959400 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:32.959425 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:32.959593 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:32.959601 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.961155 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.961159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:32.962698 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:32.962711 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.963750 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.963754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:32.963852 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:32.963875 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:17:32.963960 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 03:17:32.964037 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":206,"spinlock_wait_cycles":15488,"thread_start_us":96,"threads_started":1}
I20260812 03:17:32.964488 18485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:32.958171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:33529 (local address 127.0.0.1:60884)
0812 03:17:32.958710 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:32.958726 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:32.958818 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:32.959628 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:32.959631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:32.959644 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:32.959766 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:32.959773 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.960921 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.960928 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:32.962699 (+ 1771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:32.962711 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.963602 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.963606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:32.963695 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:32.963786 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 03:17:32.963899 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 03:17:32.964034 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":140,"thread_start_us":329,"threads_started":1}
I20260812 03:17:32.964282 18487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:32.958692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60884 (local address 127.16.217.66:33529)
0812 03:17:32.959107 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:32.959122 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:32.959149 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:17:32.959233 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:32.959239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:32.959434 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:32.959918 (+ 484us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:32.960063 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.960776 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.962873 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.963457 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.963781 (+ 324us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:32.963935 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:32.964001 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 03:17:32.964099 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":133,"threads_started":1}
I20260812 03:17:32.964594 18482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:32.958267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45306 (local address 127.16.217.67:44295)
0812 03:17:32.959126 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:32.959137 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:32.959156 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:17:32.959239 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:32.959244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:32.959410 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:32.959562 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:32.959861 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.961019 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.962563 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.963371 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.963490 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:32.963641 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:32.964311 (+ 670us) server_negotiation.cc:299] Negotiation successful
0812 03:17:32.964419 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":767,"thread_start_us":173,"threads_started":1}
I20260812 03:17:32.964905 18479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:32.957605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:44295 (local address 127.0.0.1:45306)
0812 03:17:32.958001 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:32.958055 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:32.958202 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:32.959561 (+ 1359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:32.959566 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:32.959592 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:32.959754 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:32.959763 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.961175 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.961178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:32.962436 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:32.962446 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:32.963528 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:32.963534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:32.963678 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:32.964088 (+ 410us) client_negotiation.cc:769] Sending connection context
0812 03:17:32.964166 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:17:32.964232 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":144,"spinlock_wait_cycles":1792,"thread_start_us":111,"threads_started":1}
I20260812 03:17:32.968295 18063 tablet_service.cc:1467] Processing CreateTablet for tablet bb37ba1f533f4213a84ee37ab090b2ff (DEFAULT_TABLE table=test-table [id=b576fb129ab64129a32fc35dbb9b8c0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:17:32.968288 17805 tablet_service.cc:1467] Processing CreateTablet for tablet bb37ba1f533f4213a84ee37ab090b2ff (DEFAULT_TABLE table=test-table [id=b576fb129ab64129a32fc35dbb9b8c0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:17:32.968290 18334 tablet_service.cc:1467] Processing CreateTablet for tablet bb37ba1f533f4213a84ee37ab090b2ff (DEFAULT_TABLE table=test-table [id=b576fb129ab64129a32fc35dbb9b8c0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:17:32.969338 18063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb37ba1f533f4213a84ee37ab090b2ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:32.969411 17805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb37ba1f533f4213a84ee37ab090b2ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:32.969411 18334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb37ba1f533f4213a84ee37ab090b2ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:32.976109 18490 tablet_bootstrap.cc:492] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0: Bootstrap starting.
I20260812 03:17:32.978452 18490 tablet_bootstrap.cc:654] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:32.978590 18493 tablet_bootstrap.cc:492] T bb37ba1f533f4213a84ee37ab090b2ff P b217dde889de4e4fa8eaa18bfa9e2f11: Bootstrap starting.
I20260812 03:17:32.979063 18495 tablet_bootstrap.cc:492] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b: Bootstrap starting.
I20260812 03:17:32.979925 18490 log.cc:824] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:32.981523 18493 tablet_bootstrap.cc:654] T bb37ba1f533f4213a84ee37ab090b2ff P b217dde889de4e4fa8eaa18bfa9e2f11: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:32.982396 18493 log.cc:824] T bb37ba1f533f4213a84ee37ab090b2ff P b217dde889de4e4fa8eaa18bfa9e2f11: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:32.982585 18495 tablet_bootstrap.cc:654] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:32.983834 18490 tablet_bootstrap.cc:492] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0: No bootstrap required, opened a new log
I20260812 03:17:32.983793 18495 log.cc:824] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:32.984066 18490 ts_tablet_manager.cc:1397] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260812 03:17:32.984495 18493 tablet_bootstrap.cc:492] T bb37ba1f533f4213a84ee37ab090b2ff P b217dde889de4e4fa8eaa18bfa9e2f11: No bootstrap required, opened a new log
I20260812 03:17:32.984694 18493 ts_tablet_manager.cc:1397] T bb37ba1f533f4213a84ee37ab090b2ff P b217dde889de4e4fa8eaa18bfa9e2f11: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:17:32.986300 18495 tablet_bootstrap.cc:492] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b: No bootstrap required, opened a new log
I20260812 03:17:32.986512 18495 ts_tablet_manager.cc:1397] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 03:17:32.987515 18490 raft_consensus.cc:348] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b217dde889de4e4fa8eaa18bfa9e2f11" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 39207 } } peers { permanent_uuid: "3dd235a5f73249319feaca5c870ad46b" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44295 } } peers { permanent_uuid: "21f79e6e10f44ed893d8308ef924d9a0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33529 } }
I20260812 03:17:32.987772 18490 raft_consensus.cc:374] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:32.987825 18490 raft_consensus.cc:729] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21f79e6e10f44ed893d8308ef924d9a0, State: Initialized, Role: FOLLOWER
I20260812 03:17:32.988284 18490 consensus_queue.cc:260] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b217dde889de4e4fa8eaa18bfa9e2f11" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 39207 } } peers { permanent_uuid: "3dd235a5f73249319feaca5c870ad46b" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44295 } } peers { permanent_uuid: "21f79e6e10f44ed893d8308ef924d9a0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33529 } }
I20260812 03:17:32.989573 18490 ts_tablet_manager.cc:1428] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260812 03:17:32.989686 18493 raft_consensus.cc:348] T bb37ba1f533f4213a84ee37ab090b2ff P b217dde889de4e4fa8eaa18bfa9e2f11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b217dde889de4e4fa8eaa18bfa9e2f11" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 39207 } } peers { permanent_uuid: "3dd235a5f73249319feaca5c870ad46b" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44295 } } peers { permanent_uuid: "21f79e6e10f44ed893d8308ef924d9a0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33529 } }
I20260812 03:17:32.989998 18493 raft_consensus.cc:374] T bb37ba1f533f4213a84ee37ab090b2ff P b217dde889de4e4fa8eaa18bfa9e2f11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:32.990003 18190 heartbeater.cc:503] Master 127.16.217.126:36581 was elected leader, sending a full tablet report...
I20260812 03:17:32.990067 18493 raft_consensus.cc:729] T bb37ba1f533f4213a84ee37ab090b2ff P b217dde889de4e4fa8eaa18bfa9e2f11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b217dde889de4e4fa8eaa18bfa9e2f11, State: Initialized, Role: FOLLOWER
I20260812 03:17:32.990594 18493 consensus_queue.cc:260] T bb37ba1f533f4213a84ee37ab090b2ff P b217dde889de4e4fa8eaa18bfa9e2f11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b217dde889de4e4fa8eaa18bfa9e2f11" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 39207 } } peers { permanent_uuid: "3dd235a5f73249319feaca5c870ad46b" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44295 } } peers { permanent_uuid: "21f79e6e10f44ed893d8308ef924d9a0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33529 } }
I20260812 03:17:32.990859 18495 raft_consensus.cc:348] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b217dde889de4e4fa8eaa18bfa9e2f11" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 39207 } } peers { permanent_uuid: "3dd235a5f73249319feaca5c870ad46b" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44295 } } peers { permanent_uuid: "21f79e6e10f44ed893d8308ef924d9a0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33529 } }
I20260812 03:17:32.991178 18495 raft_consensus.cc:374] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:32.991279 18495 raft_consensus.cc:729] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dd235a5f73249319feaca5c870ad46b, State: Initialized, Role: FOLLOWER
I20260812 03:17:32.991771 18495 consensus_queue.cc:260] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b217dde889de4e4fa8eaa18bfa9e2f11" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 39207 } } peers { permanent_uuid: "3dd235a5f73249319feaca5c870ad46b" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44295 } } peers { permanent_uuid: "21f79e6e10f44ed893d8308ef924d9a0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33529 } }
I20260812 03:17:32.991962 18493 ts_tablet_manager.cc:1428] T bb37ba1f533f4213a84ee37ab090b2ff P b217dde889de4e4fa8eaa18bfa9e2f11: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 03:17:32.992359 17930 heartbeater.cc:503] Master 127.16.217.126:36581 was elected leader, sending a full tablet report...
I20260812 03:17:32.992712 18495 ts_tablet_manager.cc:1428] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:17:32.992769 18461 heartbeater.cc:503] Master 127.16.217.126:36581 was elected leader, sending a full tablet report...
I20260812 03:17:33.138348 18504 raft_consensus.cc:482] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:17:33.138592 18504 raft_consensus.cc:504] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b217dde889de4e4fa8eaa18bfa9e2f11" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 39207 } } peers { permanent_uuid: "3dd235a5f73249319feaca5c870ad46b" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44295 } } peers { permanent_uuid: "21f79e6e10f44ed893d8308ef924d9a0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33529 } }
I20260812 03:17:33.139971 18504 leader_election.cc:288] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b217dde889de4e4fa8eaa18bfa9e2f11 (127.16.217.65:39207), 21f79e6e10f44ed893d8308ef924d9a0 (127.16.217.66:33529)
I20260812 03:17:33.144785 18465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:33.140372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:39207 (local address 127.16.217.67:32849)
0812 03:17:33.140518 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:33.140562 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:33.140691 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:33.141147 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:33.141150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:33.141167 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:33.141355 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:33.141370 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:33.142412 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:33.142416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:33.143371 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:33.143381 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:33.144295 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:33.144298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:33.144409 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:33.144436 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 03:17:33.144519 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 03:17:33.144591 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":3}
I20260812 03:17:33.144806 18480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:33.140498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:32849 (local address 127.16.217.65:39207)
0812 03:17:33.140662 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:33.140667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:33.140683 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:17:33.140863 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:33.140866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:33.140922 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:33.140992 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:33.141506 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:33.142181 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:33.143516 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:33.144162 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:33.144258 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:33.144302 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:33.144564 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 03:17:33.144636 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 03:17:33.145334 18487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:33.140856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:50053 (local address 127.16.217.66:33529)
0812 03:17:33.141001 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:33.141005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:33.141362 (+ 357us) server_negotiation.cc:407] Connection header received
0812 03:17:33.141429 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:33.141432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:33.141496 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:33.141573 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:33.141905 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:33.142623 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:33.144060 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:33.144623 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:33.144757 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:33.144808 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:33.145098 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 03:17:33.145176 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:17:33.145586 17839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb37ba1f533f4213a84ee37ab090b2ff" candidate_uuid: "3dd235a5f73249319feaca5c870ad46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b217dde889de4e4fa8eaa18bfa9e2f11" is_pre_election: true
I20260812 03:17:33.145675 18506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:33.140716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:33529 (local address 127.16.217.67:50053)
0812 03:17:33.141207 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:33.141221 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:33.141325 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:33.141625 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:33.141628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:33.141649 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:33.141783 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:33.141791 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:33.142799 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:33.142802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:33.143919 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:33.143930 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:33.144786 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:33.144790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:33.144905 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:33.144933 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 03:17:33.145497 (+ 564us) client_negotiation.cc:240] Negotiation successful
0812 03:17:33.145574 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":25,"thread_start_us":153,"threads_started":1}
I20260812 03:17:33.146041 17839 raft_consensus.cc:2454] T bb37ba1f533f4213a84ee37ab090b2ff P b217dde889de4e4fa8eaa18bfa9e2f11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3dd235a5f73249319feaca5c870ad46b in term 0.
I20260812 03:17:33.146406 18099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb37ba1f533f4213a84ee37ab090b2ff" candidate_uuid: "3dd235a5f73249319feaca5c870ad46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21f79e6e10f44ed893d8308ef924d9a0" is_pre_election: true
I20260812 03:17:33.146727 18099 raft_consensus.cc:2454] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3dd235a5f73249319feaca5c870ad46b in term 0.
I20260812 03:17:33.146716 18246 leader_election.cc:302] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3dd235a5f73249319feaca5c870ad46b, b217dde889de4e4fa8eaa18bfa9e2f11; no voters:
I20260812 03:17:33.147032 18504 raft_consensus.cc:2781] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:17:33.147142 18504 raft_consensus.cc:482] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260812 03:17:33.147156 18191 tablet.cc:2369] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:17:33.147202 18504 raft_consensus.cc:3037] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:33.148399 18504 raft_consensus.cc:504] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b217dde889de4e4fa8eaa18bfa9e2f11" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 39207 } } peers { permanent_uuid: "3dd235a5f73249319feaca5c870ad46b" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44295 } } peers { permanent_uuid: "21f79e6e10f44ed893d8308ef924d9a0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33529 } }
I20260812 03:17:33.148753 18504 leader_election.cc:288] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [CANDIDATE]: Term 1 election: Requested vote from peers b217dde889de4e4fa8eaa18bfa9e2f11 (127.16.217.65:39207), 21f79e6e10f44ed893d8308ef924d9a0 (127.16.217.66:33529)
I20260812 03:17:33.149176 17839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb37ba1f533f4213a84ee37ab090b2ff" candidate_uuid: "3dd235a5f73249319feaca5c870ad46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b217dde889de4e4fa8eaa18bfa9e2f11"
I20260812 03:17:33.149257 18099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb37ba1f533f4213a84ee37ab090b2ff" candidate_uuid: "3dd235a5f73249319feaca5c870ad46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21f79e6e10f44ed893d8308ef924d9a0"
I20260812 03:17:33.149322 17839 raft_consensus.cc:3037] T bb37ba1f533f4213a84ee37ab090b2ff P b217dde889de4e4fa8eaa18bfa9e2f11 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:33.149390 18099 raft_consensus.cc:3037] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:33.150484 18099 raft_consensus.cc:2454] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3dd235a5f73249319feaca5c870ad46b in term 1.
I20260812 03:17:33.150484 17839 raft_consensus.cc:2454] T bb37ba1f533f4213a84ee37ab090b2ff P b217dde889de4e4fa8eaa18bfa9e2f11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3dd235a5f73249319feaca5c870ad46b in term 1.
I20260812 03:17:33.150871 18246 leader_election.cc:302] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3dd235a5f73249319feaca5c870ad46b, b217dde889de4e4fa8eaa18bfa9e2f11; no voters:
I20260812 03:17:33.151069 18504 raft_consensus.cc:2781] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:33.151330 18504 raft_consensus.cc:686] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [term 1 LEADER]: Becoming Leader. State: Replica: 3dd235a5f73249319feaca5c870ad46b, State: Running, Role: LEADER
I20260812 03:17:33.151696 18504 consensus_queue.cc:237] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b217dde889de4e4fa8eaa18bfa9e2f11" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 39207 } } peers { permanent_uuid: "3dd235a5f73249319feaca5c870ad46b" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44295 } } peers { permanent_uuid: "21f79e6e10f44ed893d8308ef924d9a0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33529 } }
I20260812 03:17:33.154892 17558 catalog_manager.cc:5495] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3dd235a5f73249319feaca5c870ad46b (127.16.217.67). New cstate: current_term: 1 leader_uuid: "3dd235a5f73249319feaca5c870ad46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b217dde889de4e4fa8eaa18bfa9e2f11" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 39207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3dd235a5f73249319feaca5c870ad46b" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21f79e6e10f44ed893d8308ef924d9a0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33529 } health_report { overall_health: UNKNOWN } } }
W20260812 03:17:33.160902 18462 tablet.cc:2369] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:17:33.163056 17932 tablet.cc:2369] T bb37ba1f533f4213a84ee37ab090b2ff P b217dde889de4e4fa8eaa18bfa9e2f11: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:17:33.204185 18482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:33.200348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45308 (local address 127.16.217.67:44295)
0812 03:17:33.200485 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:33.200488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:33.200501 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:17:33.200572 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:33.200575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:33.200630 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:33.200725 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:17:33.200936 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:33.201429 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:33.202433 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:33.202647 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:33.202732 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:33.203134 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:17:33.203543 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:17:33.203663 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:33.203977 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 03:17:33.204018 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:17:33.210326 17839 raft_consensus.cc:1261] T bb37ba1f533f4213a84ee37ab090b2ff P b217dde889de4e4fa8eaa18bfa9e2f11 [term 1 FOLLOWER]: Refusing update from remote peer 3dd235a5f73249319feaca5c870ad46b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:17:33.210359 18099 raft_consensus.cc:1261] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0 [term 1 FOLLOWER]: Refusing update from remote peer 3dd235a5f73249319feaca5c870ad46b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:17:33.211143 18504 consensus_queue.cc:1035] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b217dde889de4e4fa8eaa18bfa9e2f11" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 39207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:17:33.211402 18508 consensus_queue.cc:1035] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "21f79e6e10f44ed893d8308ef924d9a0" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 33529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:17:33.218655 18512 mvcc.cc:205] Tried to move back new op lower bound from 7317523059541659648 to 7317523059311828992. Current Snapshot: MvccSnapshot[applied={T|T < 7317523059541659648}]
I20260812 03:17:33.228444 17559 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:17:33.228623 17559 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:17:33.231145 17559 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89cb5d96cd7e4a90975fed56f2629920: Sending DeleteTablet for 3 replicas of tablet bb37ba1f533f4213a84ee37ab090b2ff
I20260812 03:17:33.231945 17805 tablet_service.cc:1514] Processing DeleteTablet for tablet bb37ba1f533f4213a84ee37ab090b2ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:17:33 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:56638
I20260812 03:17:33.232110 18334 tablet_service.cc:1514] Processing DeleteTablet for tablet bb37ba1f533f4213a84ee37ab090b2ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:17:33 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:45306
I20260812 03:17:33.232121 18063 tablet_service.cc:1514] Processing DeleteTablet for tablet bb37ba1f533f4213a84ee37ab090b2ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:17:33 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:60884
I20260812 03:17:33.232652 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 17683
I20260812 03:17:33.240448 18532 tablet_replica.cc:330] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b: stopping tablet replica
I20260812 03:17:33.240806 18532 raft_consensus.cc:2229] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:17:33.241078 18533 tablet_replica.cc:330] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0: stopping tablet replica
I20260812 03:17:33.241417 18533 raft_consensus.cc:2229] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:17:33.241847 18533 raft_consensus.cc:2258] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:17:33.242338 18532 raft_consensus.cc:2258] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:17:33.244365 18533 ts_tablet_manager.cc:1905] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:17:33.245347 18532 ts_tablet_manager.cc:1905] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:17:33.248214 18532 ts_tablet_manager.cc:1918] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:17:33.248380 18532 log.cc:1196] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/bb37ba1f533f4213a84ee37ab090b2ff
I20260812 03:17:33.248790 18532 ts_tablet_manager.cc:1939] T bb37ba1f533f4213a84ee37ab090b2ff P 3dd235a5f73249319feaca5c870ad46b: Deleting consensus metadata
I20260812 03:17:33.249660 18533 ts_tablet_manager.cc:1918] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:17:33.249830 18533 log.cc:1196] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/bb37ba1f533f4213a84ee37ab090b2ff
W20260812 03:17:33.249828 17536 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.217.65:39207 (error 108)
I20260812 03:17:33.250274 18533 ts_tablet_manager.cc:1939] T bb37ba1f533f4213a84ee37ab090b2ff P 21f79e6e10f44ed893d8308ef924d9a0: Deleting consensus metadata
I20260812 03:17:33.250466 17533 catalog_manager.cc:4841] TS 3dd235a5f73249319feaca5c870ad46b (127.16.217.67:44295): tablet bb37ba1f533f4213a84ee37ab090b2ff (table test-table [id=b576fb129ab64129a32fc35dbb9b8c0f]) successfully deleted
I20260812 03:17:33.251350 17536 catalog_manager.cc:4841] TS 21f79e6e10f44ed893d8308ef924d9a0 (127.16.217.66:33529): tablet bb37ba1f533f4213a84ee37ab090b2ff (table test-table [id=b576fb129ab64129a32fc35dbb9b8c0f]) successfully deleted
W20260812 03:17:33.251507 18479 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:17:33.250857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:39207 (local address 127.0.0.1:56640)
0812 03:17:33.251029 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:33.251045 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:33.251131 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:33.251362 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:39207: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":95}
W20260812 03:17:33.251662 17536 catalog_manager.cc:4584] TS b217dde889de4e4fa8eaa18bfa9e2f11 (127.16.217.65:39207): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bb37ba1f533f4213a84ee37ab090b2ff: Network error: Client connection negotiation failed: client connection to 127.16.217.65:39207: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 03:17:33.251791 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 17946
I20260812 03:17:33.270010 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 18201
I20260812 03:17:33.285651 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 17505
2026-08-12T03:17:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2866 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T03:17:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:17:33Z Disabled control of system clock
I20260812 03:17:33.327976 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:44117
--webserver_interface=127.16.217.126
--webserver_port=0
--builtin_ntp_servers=127.16.217.84:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:44117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:17:33.449586 18541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:33.449867 18541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:33.449908 18541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:33.455638 18541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:33.455761 18541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:33.455780 18541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:33.455799 18541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:33.455817 18541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:33.463583 18541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:44117
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:44117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:33.464915 18541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:33.466472 18541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:33.475934 18550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:33.476153 18552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:33.476018 18541 server_base.cc:1034] running on GCE node
W20260812 03:17:33.476270 18554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:33.476915 18541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:33.478080 18541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:33.479277 18541 hybrid_clock.cc:648] HybridClock initialized: now 1786504653479225 us; error 63 us; skew 500 ppm
I20260812 03:17:33.481551 18541 webserver.cc:460] Webserver started at http://127.16.217.126:36675/ using document root <none> and password file <none>
I20260812 03:17:33.482209 18541 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:33.482340 18541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:33.482587 18541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:33.484817 18541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "396a89a4075d41c980d9494119ed4ac3"
format_stamp: "Formatted at 2026-08-12 03:17:33 on dist-test-slave-7nm7"
server_key: "510a4c656432daf3b5477e79527f3d9a"
server_key_iv: "0447ddf41b7700dc6e3e0020ee25d806"
server_key_version: "encryptionkey@0"
I20260812 03:17:33.485422 18541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "396a89a4075d41c980d9494119ed4ac3"
format_stamp: "Formatted at 2026-08-12 03:17:33 on dist-test-slave-7nm7"
server_key: "510a4c656432daf3b5477e79527f3d9a"
server_key_iv: "0447ddf41b7700dc6e3e0020ee25d806"
server_key_version: "encryptionkey@0"
I20260812 03:17:33.490118 18541 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:17:33.493435 18563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:33.495163 18541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:17:33.495344 18541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "396a89a4075d41c980d9494119ed4ac3"
format_stamp: "Formatted at 2026-08-12 03:17:33 on dist-test-slave-7nm7"
server_key: "510a4c656432daf3b5477e79527f3d9a"
server_key_iv: "0447ddf41b7700dc6e3e0020ee25d806"
server_key_version: "encryptionkey@0"
I20260812 03:17:33.495476 18541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:33.522548 18541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:33.523365 18541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:33.523588 18541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:33.531836 18541 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:44117
I20260812 03:17:33.531836 18668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:44117 every 8 connection(s)
I20260812 03:17:33.532944 18541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:17:33.535684 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 18541
I20260812 03:17:33.535861 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:17:33.536420 17253 external_mini_cluster.cc:1428] Setting key 7b20664f4e18f0d99f6d5453785517b0
I20260812 03:17:33.537254 18669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:33.544368 18669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3: Bootstrap starting.
I20260812 03:17:33.546875 18669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:33.546953 18683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:33.538104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36580 (local address 127.16.217.126:44117)
0812 03:17:33.538825 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:33.538840 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:33.538876 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:17:33.539968 (+ 1092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:33.539987 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:33.540402 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:33.540703 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:33.541124 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:33.542080 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:33.543369 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:33.543837 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:33.543957 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:33.545094 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:33.545100 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:33.545126 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:33.545527 (+ 401us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:33.545579 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:33.545599 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:33.545717 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:33.545852 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 03:17:33.546131 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":225,"threads_started":1}
I20260812 03:17:33.547710 18669 log.cc:824] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:33.550014 18669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3: No bootstrap required, opened a new log
I20260812 03:17:33.554070 18669 raft_consensus.cc:348] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396a89a4075d41c980d9494119ed4ac3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44117 } }
I20260812 03:17:33.554371 18669 raft_consensus.cc:374] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:33.554459 18669 raft_consensus.cc:729] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396a89a4075d41c980d9494119ed4ac3, State: Initialized, Role: FOLLOWER
I20260812 03:17:33.554905 18669 consensus_queue.cc:260] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396a89a4075d41c980d9494119ed4ac3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44117 } }
I20260812 03:17:33.555030 18669 raft_consensus.cc:388] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:17:33.555115 18669 raft_consensus.cc:482] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:17:33.555256 18669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:33.556249 18669 raft_consensus.cc:504] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396a89a4075d41c980d9494119ed4ac3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44117 } }
I20260812 03:17:33.556612 18669 leader_election.cc:302] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 396a89a4075d41c980d9494119ed4ac3; no voters:
I20260812 03:17:33.556972 18669 leader_election.cc:288] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:17:33.557261 18687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:33.557497 18687 raft_consensus.cc:686] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3 [term 1 LEADER]: Becoming Leader. State: Replica: 396a89a4075d41c980d9494119ed4ac3, State: Running, Role: LEADER
I20260812 03:17:33.557823 18687 consensus_queue.cc:237] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396a89a4075d41c980d9494119ed4ac3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44117 } }
I20260812 03:17:33.558354 18669 sys_catalog.cc:564] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:17:33.559635 18688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "396a89a4075d41c980d9494119ed4ac3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396a89a4075d41c980d9494119ed4ac3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44117 } } }
I20260812 03:17:33.559762 18688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:33.560184 18691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 396a89a4075d41c980d9494119ed4ac3. Latest consensus state: current_term: 1 leader_uuid: "396a89a4075d41c980d9494119ed4ac3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396a89a4075d41c980d9494119ed4ac3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44117 } } }
I20260812 03:17:33.560355 18691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:33.560302 18704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:17:33.563429 18704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:17:33.569126 18704 catalog_manager.cc:1347] Generated new cluster ID: 48ab530a29e948b6be53d037b4fb20d4
I20260812 03:17:33.569216 18704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:17:33.574615 18704 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:17:33.575503 18704 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:17:33.586215 18704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3: Generated new TSK 0
I20260812 03:17:33.586956 18704 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:17:33.591409 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:44117
--builtin_ntp_servers=127.16.217.84:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:17:33.714407 18721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:33.714745 18721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:33.714819 18721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:33.719090 18721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:33.719177 18721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:33.719267 18721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:17:33.724300 18721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:44117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:33.725602 18721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:33.727167 18721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:33.737689 18732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:33.737738 18730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:33.737754 18728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:33.738035 18721 server_base.cc:1034] running on GCE node
I20260812 03:17:33.738559 18721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:33.739344 18721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:33.740581 18721 hybrid_clock.cc:648] HybridClock initialized: now 1786504653740561 us; error 30 us; skew 500 ppm
I20260812 03:17:33.743283 18721 webserver.cc:460] Webserver started at http://127.16.217.65:46307/ using document root <none> and password file <none>
I20260812 03:17:33.743950 18721 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:33.744051 18721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:33.744304 18721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:33.746627 18721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "11391fbc35b244b68d8579773ed9026b"
format_stamp: "Formatted at 2026-08-12 03:17:33 on dist-test-slave-7nm7"
server_key: "8b45fea590622a394279293775818cf8"
server_key_iv: "d0b3f0932458614e439d54b629a15670"
server_key_version: "encryptionkey@0"
I20260812 03:17:33.747239 18721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "11391fbc35b244b68d8579773ed9026b"
format_stamp: "Formatted at 2026-08-12 03:17:33 on dist-test-slave-7nm7"
server_key: "8b45fea590622a394279293775818cf8"
server_key_iv: "d0b3f0932458614e439d54b629a15670"
server_key_version: "encryptionkey@0"
I20260812 03:17:33.751199 18721 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:17:33.753911 18742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:33.755272 18721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:17:33.755434 18721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "11391fbc35b244b68d8579773ed9026b"
format_stamp: "Formatted at 2026-08-12 03:17:33 on dist-test-slave-7nm7"
server_key: "8b45fea590622a394279293775818cf8"
server_key_iv: "d0b3f0932458614e439d54b629a15670"
server_key_version: "encryptionkey@0"
I20260812 03:17:33.755563 18721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:33.778455 18721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:33.780494 18721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:33.780974 18721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:33.781822 18721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:33.783424 18721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:33.783555 18721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:33.783640 18721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:33.783695 18721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:33.794153 18721 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:40043
I20260812 03:17:33.794173 18939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:40043 every 8 connection(s)
I20260812 03:17:33.795190 18721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:17:33.804096 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 18721
I20260812 03:17:33.804282 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:17:33.804726 17253 external_mini_cluster.cc:1428] Setting key a16fd48fba4800136853031d5faba6d2
I20260812 03:17:33.805683 18683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:33.797203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:44863 (local address 127.16.217.126:44117)
0812 03:17:33.797362 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:33.797365 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:33.798658 (+ 1293us) server_negotiation.cc:407] Connection header received
0812 03:17:33.800288 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:33.800292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:33.800361 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:33.800474 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:33.801415 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:33.801954 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:33.803251 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:33.803471 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:33.803592 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:33.805024 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:33.805028 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:33.805030 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:33.805044 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:33.805064 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:33.805072 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:33.805144 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:33.805443 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 03:17:33.805543 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:17:33.806720 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:33.797546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:44117 (local address 127.16.217.65:44863)
0812 03:17:33.798501 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:33.798550 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:33.800030 (+ 1480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:33.800662 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:33.800670 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:33.800756 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:33.801260 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:33.801273 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:33.802103 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:33.802106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:33.803082 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:33.803092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:33.803643 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:33.803647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:33.803899 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:33.804671 (+ 772us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:17:33.804673 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:33.804745 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:17:33.804841 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:33.805178 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:33.805199 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:33.805208 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 03:17:33.805352 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 03:17:33.805647 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":128,"threads_started":1}
I20260812 03:17:33.807231 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:44117
--builtin_ntp_servers=127.16.217.84:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:17:33.808123 18940 heartbeater.cc:347] Connected to a master server at 127.16.217.126:44117
I20260812 03:17:33.808385 18940 heartbeater.cc:464] Registering TS with master...
I20260812 03:17:33.809012 18940 heartbeater.cc:511] Master 127.16.217.126:44117 requested a full tablet report, sending...
I20260812 03:17:33.811414 18601 ts_manager.cc:194] Registered new tserver with Master: 11391fbc35b244b68d8579773ed9026b (127.16.217.65:40043)
I20260812 03:17:33.813290 18601 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.217.65:44863
W20260812 03:17:33.934198 18947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:33.934679 18947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:33.934769 18947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:33.938894 18947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:33.938970 18947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:33.939059 18947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:17:33.943912 18947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:44117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:33.945133 18947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:33.946703 18947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:33.957435 18958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:33.957414 18962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:33.957417 18956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:33.958488 18947 server_base.cc:1034] running on GCE node
I20260812 03:17:33.958992 18947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:33.959770 18947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:33.960958 18947 hybrid_clock.cc:648] HybridClock initialized: now 1786504653960943 us; error 54 us; skew 500 ppm
I20260812 03:17:33.963101 18947 webserver.cc:460] Webserver started at http://127.16.217.66:39515/ using document root <none> and password file <none>
I20260812 03:17:33.963739 18947 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:33.963810 18947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:33.964037 18947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:33.966197 18947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "cc520894e91946fda723dd96d0088a4c"
format_stamp: "Formatted at 2026-08-12 03:17:33 on dist-test-slave-7nm7"
server_key: "6e624ff7ac723730ff94f2d02c768b8f"
server_key_iv: "29b1bf483008c916e00f756fb834ee98"
server_key_version: "encryptionkey@0"
I20260812 03:17:33.966845 18947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "cc520894e91946fda723dd96d0088a4c"
format_stamp: "Formatted at 2026-08-12 03:17:33 on dist-test-slave-7nm7"
server_key: "6e624ff7ac723730ff94f2d02c768b8f"
server_key_iv: "29b1bf483008c916e00f756fb834ee98"
server_key_version: "encryptionkey@0"
I20260812 03:17:33.970609 18947 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:17:33.973150 18973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:33.974517 18947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:17:33.974671 18947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "cc520894e91946fda723dd96d0088a4c"
format_stamp: "Formatted at 2026-08-12 03:17:33 on dist-test-slave-7nm7"
server_key: "6e624ff7ac723730ff94f2d02c768b8f"
server_key_iv: "29b1bf483008c916e00f756fb834ee98"
server_key_version: "encryptionkey@0"
I20260812 03:17:33.974792 18947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:34.001030 18947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:34.001793 18947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:34.002024 18947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:34.002801 18947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:34.003831 18947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:34.003909 18947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:34.003985 18947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:34.004030 18947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:34.013489 18947 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:42051
I20260812 03:17:34.013579 19193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:42051 every 8 connection(s)
I20260812 03:17:34.014603 18947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:17:34.016217 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 18947
I20260812 03:17:34.016358 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:17:34.016669 17253 external_mini_cluster.cc:1428] Setting key 444865dd86581d1ad5bed8fa065ca1a5
I20260812 03:17:34.018853 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:44117
--builtin_ntp_servers=127.16.217.84:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:17:34.027050 18683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:34.016842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:57207 (local address 127.16.217.126:44117)
0812 03:17:34.017009 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:34.017015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:34.018340 (+ 1325us) server_negotiation.cc:407] Connection header received
0812 03:17:34.020337 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:34.020344 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:34.020410 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:34.020507 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:34.021872 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.023113 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.024373 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.024841 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.024971 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:34.026394 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:34.026401 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:34.026406 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:34.026427 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:34.026454 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:34.026465 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:34.026565 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:34.026749 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 03:17:34.026829 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:17:34.027840 19201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:34.017381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:44117 (local address 127.16.217.66:57207)
0812 03:17:34.018156 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:34.018209 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:34.020081 (+ 1872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:34.020746 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:34.020758 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:34.020886 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:34.021656 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:34.021677 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.023297 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.023301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:34.024232 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:34.024242 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.025072 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.025077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:34.025242 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:34.025904 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:17:34.025906 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:34.025950 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:17:34.026020 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:34.026581 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:34.026589 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:34.026595 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:17:34.026709 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 03:17:34.026947 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":167,"threads_started":1}
I20260812 03:17:34.029207 19194 heartbeater.cc:347] Connected to a master server at 127.16.217.126:44117
I20260812 03:17:34.029462 19194 heartbeater.cc:464] Registering TS with master...
I20260812 03:17:34.030087 19194 heartbeater.cc:511] Master 127.16.217.126:44117 requested a full tablet report, sending...
I20260812 03:17:34.031350 18601 ts_manager.cc:194] Registered new tserver with Master: cc520894e91946fda723dd96d0088a4c (127.16.217.66:42051)
I20260812 03:17:34.031983 18601 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.217.66:57207
W20260812 03:17:34.196758 19202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:34.197085 19202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:34.197167 19202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:34.201542 19202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:34.201632 19202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:34.201725 19202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:17:34.207515 19202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:44117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:34.209043 19202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:34.210660 19202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:34.221618 19210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:34.221776 19211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:34.221635 19214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:34.221673 19202 server_base.cc:1034] running on GCE node
I20260812 03:17:34.223163 19202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:34.224282 19202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:34.225593 19202 hybrid_clock.cc:648] HybridClock initialized: now 1786504654225560 us; error 37 us; skew 500 ppm
I20260812 03:17:34.227907 19202 webserver.cc:460] Webserver started at http://127.16.217.67:38651/ using document root <none> and password file <none>
I20260812 03:17:34.228562 19202 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:34.228670 19202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:34.228924 19202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:34.231176 19202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "ef61e13feedf4dccac963fe18f1e833e"
format_stamp: "Formatted at 2026-08-12 03:17:34 on dist-test-slave-7nm7"
server_key: "3b69e79d9ec9d1b919c7f99c507870f1"
server_key_iv: "60b5432a80e47a39bb2b972263079f36"
server_key_version: "encryptionkey@0"
I20260812 03:17:34.231791 19202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "ef61e13feedf4dccac963fe18f1e833e"
format_stamp: "Formatted at 2026-08-12 03:17:34 on dist-test-slave-7nm7"
server_key: "3b69e79d9ec9d1b919c7f99c507870f1"
server_key_iv: "60b5432a80e47a39bb2b972263079f36"
server_key_version: "encryptionkey@0"
I20260812 03:17:34.235952 19202 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:17:34.238660 19226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:34.239948 19202 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 03:17:34.240099 19202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "ef61e13feedf4dccac963fe18f1e833e"
format_stamp: "Formatted at 2026-08-12 03:17:34 on dist-test-slave-7nm7"
server_key: "3b69e79d9ec9d1b919c7f99c507870f1"
server_key_iv: "60b5432a80e47a39bb2b972263079f36"
server_key_version: "encryptionkey@0"
I20260812 03:17:34.240232 19202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:34.260435 19202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:34.261258 19202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:34.261493 19202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:34.262171 19202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:34.263299 19202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:34.263381 19202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:34.263464 19202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:34.263521 19202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:34.273682 19202 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:46061
I20260812 03:17:34.273720 19446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:46061 every 8 connection(s)
I20260812 03:17:34.274716 19202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:17:34.276690 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 19202
I20260812 03:17:34.276801 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:17:34.277112 17253 external_mini_cluster.cc:1428] Setting key 1143cdb7b4e3fb9333edd3b67a525adb
I20260812 03:17:34.285185 18683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:34.276924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:49419 (local address 127.16.217.126:44117)
0812 03:17:34.277067 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:34.277071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:34.278709 (+ 1638us) server_negotiation.cc:407] Connection header received
0812 03:17:34.280263 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:34.280266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:34.280322 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:34.280427 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:34.281316 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.281842 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.283013 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.283234 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.283483 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:34.284661 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:34.284665 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:34.284668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:34.284681 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:34.284702 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:34.284726 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:34.284804 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:34.284968 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 03:17:34.285030 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:17:34.286017 19450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:34.277424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:44117 (local address 127.16.217.67:49419)
0812 03:17:34.278492 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:34.278550 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:34.280047 (+ 1497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:34.280599 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:34.280607 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:34.280686 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:34.281165 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:34.281178 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.281967 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.281971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:34.282880 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:34.282889 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.283521 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.283526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:34.283713 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:34.284392 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:17:34.284393 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:34.284433 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:17:34.284498 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:34.284818 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:34.284831 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:34.284841 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 03:17:34.284968 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 03:17:34.285247 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":166,"threads_started":1}
I20260812 03:17:34.287289 19447 heartbeater.cc:347] Connected to a master server at 127.16.217.126:44117
I20260812 03:17:34.287493 19447 heartbeater.cc:464] Registering TS with master...
I20260812 03:17:34.287976 19447 heartbeater.cc:511] Master 127.16.217.126:44117 requested a full tablet report, sending...
I20260812 03:17:34.288980 18601 ts_manager.cc:194] Registered new tserver with Master: ef61e13feedf4dccac963fe18f1e833e (127.16.217.67:46061)
I20260812 03:17:34.289577 18601 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.217.67:49419
I20260812 03:17:34.291297 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:17:34.297339 18683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:34.293375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47398 (local address 127.16.217.126:44117)
0812 03:17:34.293513 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:34.293517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:34.293681 (+ 164us) server_negotiation.cc:407] Connection header received
0812 03:17:34.293774 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:34.293776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:34.293814 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:34.293880 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:34.294172 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.294771 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.295591 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.295805 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.295908 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:34.296865 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:34.296869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:34.296871 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:34.296884 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:34.296903 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:34.296912 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:34.297019 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:34.297119 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 03:17:34.297152 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:17:34.300673 18601 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:17:34.303947 18601 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:17:34.319087 19477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:34.313612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59396 (local address 127.16.217.67:46061)
0812 03:17:34.314225 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:34.314257 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:34.314277 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:17:34.314342 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:34.314347 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:34.314486 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:34.314934 (+ 448us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:34.315078 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.315892 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.317397 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.318221 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.318365 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:34.318575 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:34.318796 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 03:17:34.318903 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":136,"threads_started":1}
I20260812 03:17:34.319079 19479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:34.313719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:40043 (local address 127.0.0.1:41272)
0812 03:17:34.314056 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:34.314069 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:34.314162 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:34.314900 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:34.314904 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:34.314925 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:34.315015 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:34.315020 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.316560 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.316563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:34.317693 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:34.317704 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.318610 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.318613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:34.318708 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:34.318737 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 03:17:34.318782 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 03:17:34.318895 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":4,"thread_start_us":114,"threads_started":1}
I20260812 03:17:34.319079 19476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:34.313112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:46061 (local address 127.0.0.1:59396)
0812 03:17:34.313699 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:34.313716 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:34.313819 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:34.314663 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:34.314669 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:34.314698 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:34.314954 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:34.314962 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.316022 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.316028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:34.317242 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:34.317253 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.318420 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.318426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:34.318564 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:34.318619 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 03:17:34.318733 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 03:17:34.318895 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":69,"thread_start_us":111,"threads_started":1}
I20260812 03:17:34.319499 19468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:34.312886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:42051 (local address 127.0.0.1:43924)
0812 03:17:34.313271 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:34.313319 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:34.313451 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:34.314816 (+ 1365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:34.314818 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:34.314832 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:34.315090 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:34.315096 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.316569 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.316574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:34.317732 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:34.317742 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.319040 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.319044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:34.319165 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:34.319199 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 03:17:34.319288 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 03:17:34.319376 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":90,"thread_start_us":101,"threads_started":1}
I20260812 03:17:34.319957 19481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:34.313879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41272 (local address 127.16.217.65:40043)
0812 03:17:34.314421 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:34.314428 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:34.314451 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:17:34.314530 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:34.314535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:34.314745 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:34.315132 (+ 387us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:34.315288 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.316417 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.317818 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.318467 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.319489 (+ 1022us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:34.319715 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:34.319767 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 03:17:34.319858 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":96,"threads_started":1}
I20260812 03:17:34.319990 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:34.313045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43924 (local address 127.16.217.66:42051)
0812 03:17:34.314095 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:34.314107 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:34.314135 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:17:34.314220 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:34.314226 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:34.314471 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:34.314637 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:34.315225 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.316438 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.317856 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.318672 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.318804 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:34.318969 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:34.319730 (+ 761us) server_negotiation.cc:299] Negotiation successful
0812 03:17:34.319841 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":923,"thread_start_us":154,"threads_started":1}
I20260812 03:17:34.322727 19055 tablet_service.cc:1467] Processing CreateTablet for tablet 69d02ffd128a44d8aa82519e1b88fbb8 (DEFAULT_TABLE table=test-table [id=718a5baaa89244d4bb4bfbed303e5484]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:17:34.322741 18817 tablet_service.cc:1467] Processing CreateTablet for tablet 69d02ffd128a44d8aa82519e1b88fbb8 (DEFAULT_TABLE table=test-table [id=718a5baaa89244d4bb4bfbed303e5484]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:17:34.322742 19317 tablet_service.cc:1467] Processing CreateTablet for tablet 69d02ffd128a44d8aa82519e1b88fbb8 (DEFAULT_TABLE table=test-table [id=718a5baaa89244d4bb4bfbed303e5484]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:17:34.323833 19055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69d02ffd128a44d8aa82519e1b88fbb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:34.323838 19317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69d02ffd128a44d8aa82519e1b88fbb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:34.323838 18817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69d02ffd128a44d8aa82519e1b88fbb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:34.330621 19482 tablet_bootstrap.cc:492] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c: Bootstrap starting.
I20260812 03:17:34.331537 19483 tablet_bootstrap.cc:492] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b: Bootstrap starting.
I20260812 03:17:34.331756 19485 tablet_bootstrap.cc:492] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e: Bootstrap starting.
I20260812 03:17:34.332993 19482 tablet_bootstrap.cc:654] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:34.333895 19482 log.cc:824] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:34.333994 19483 tablet_bootstrap.cc:654] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:34.333994 19485 tablet_bootstrap.cc:654] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:34.334909 19483 log.cc:824] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:34.335064 19485 log.cc:824] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:34.336364 19482 tablet_bootstrap.cc:492] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c: No bootstrap required, opened a new log
I20260812 03:17:34.336591 19482 ts_tablet_manager.cc:1397] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 03:17:34.336807 19483 tablet_bootstrap.cc:492] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b: No bootstrap required, opened a new log
I20260812 03:17:34.337006 19483 ts_tablet_manager.cc:1397] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 03:17:34.337141 19485 tablet_bootstrap.cc:492] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e: No bootstrap required, opened a new log
I20260812 03:17:34.337332 19485 ts_tablet_manager.cc:1397] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260812 03:17:34.340534 19483 raft_consensus.cc:348] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc520894e91946fda723dd96d0088a4c" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42051 } } peers { permanent_uuid: "11391fbc35b244b68d8579773ed9026b" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40043 } } peers { permanent_uuid: "ef61e13feedf4dccac963fe18f1e833e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46061 } }
I20260812 03:17:34.340787 19483 raft_consensus.cc:374] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:34.340870 19483 raft_consensus.cc:729] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11391fbc35b244b68d8579773ed9026b, State: Initialized, Role: FOLLOWER
I20260812 03:17:34.340986 19482 raft_consensus.cc:348] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc520894e91946fda723dd96d0088a4c" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42051 } } peers { permanent_uuid: "11391fbc35b244b68d8579773ed9026b" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40043 } } peers { permanent_uuid: "ef61e13feedf4dccac963fe18f1e833e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46061 } }
I20260812 03:17:34.341244 19482 raft_consensus.cc:374] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:34.341353 19482 raft_consensus.cc:729] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc520894e91946fda723dd96d0088a4c, State: Initialized, Role: FOLLOWER
I20260812 03:17:34.341387 19483 consensus_queue.cc:260] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc520894e91946fda723dd96d0088a4c" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42051 } } peers { permanent_uuid: "11391fbc35b244b68d8579773ed9026b" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40043 } } peers { permanent_uuid: "ef61e13feedf4dccac963fe18f1e833e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46061 } }
I20260812 03:17:34.341423 19485 raft_consensus.cc:348] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc520894e91946fda723dd96d0088a4c" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42051 } } peers { permanent_uuid: "11391fbc35b244b68d8579773ed9026b" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40043 } } peers { permanent_uuid: "ef61e13feedf4dccac963fe18f1e833e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46061 } }
I20260812 03:17:34.341673 19485 raft_consensus.cc:374] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:34.341832 19482 consensus_queue.cc:260] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc520894e91946fda723dd96d0088a4c" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42051 } } peers { permanent_uuid: "11391fbc35b244b68d8579773ed9026b" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40043 } } peers { permanent_uuid: "ef61e13feedf4dccac963fe18f1e833e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46061 } }
I20260812 03:17:34.342103 18940 heartbeater.cc:503] Master 127.16.217.126:44117 was elected leader, sending a full tablet report...
I20260812 03:17:34.342514 19483 ts_tablet_manager.cc:1428] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 03:17:34.342864 19482 ts_tablet_manager.cc:1428] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260812 03:17:34.341781 19485 raft_consensus.cc:729] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef61e13feedf4dccac963fe18f1e833e, State: Initialized, Role: FOLLOWER
I20260812 03:17:34.343278 19194 heartbeater.cc:503] Master 127.16.217.126:44117 was elected leader, sending a full tablet report...
I20260812 03:17:34.343483 19485 consensus_queue.cc:260] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc520894e91946fda723dd96d0088a4c" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42051 } } peers { permanent_uuid: "11391fbc35b244b68d8579773ed9026b" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40043 } } peers { permanent_uuid: "ef61e13feedf4dccac963fe18f1e833e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46061 } }
I20260812 03:17:34.344168 19447 heartbeater.cc:503] Master 127.16.217.126:44117 was elected leader, sending a full tablet report...
I20260812 03:17:34.344583 19485 ts_tablet_manager.cc:1428] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
W20260812 03:17:34.515882 19197 tablet.cc:2369] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:17:34.520236 19492 raft_consensus.cc:482] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:17:34.520529 19492 raft_consensus.cc:504] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc520894e91946fda723dd96d0088a4c" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42051 } } peers { permanent_uuid: "11391fbc35b244b68d8579773ed9026b" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40043 } } peers { permanent_uuid: "ef61e13feedf4dccac963fe18f1e833e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46061 } }
I20260812 03:17:34.521768 19492 leader_election.cc:288] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc520894e91946fda723dd96d0088a4c (127.16.217.66:42051), ef61e13feedf4dccac963fe18f1e833e (127.16.217.67:46061)
W20260812 03:17:34.525701 19449 tablet.cc:2369] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:17:34.526556 19501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:34.522105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:42051 (local address 127.16.217.65:55071)
0812 03:17:34.522749 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:34.522762 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:34.522908 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:34.523203 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:34.523206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:34.523223 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:34.523490 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:34.523497 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.524272 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.524276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:34.525150 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:34.525159 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.526067 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.526071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:34.526177 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:34.526205 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 03:17:34.526317 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 03:17:34.526394 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":387,"threads_started":1}
I20260812 03:17:34.526714 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:34.522270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:55071 (local address 127.16.217.66:42051)
0812 03:17:34.522492 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:34.522499 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:34.522862 (+ 363us) server_negotiation.cc:407] Connection header received
0812 03:17:34.523036 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:34.523040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:34.523104 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:34.523191 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:34.523620 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.524145 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.525284 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.525927 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.526060 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:34.526117 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:34.526485 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 03:17:34.526564 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260812 03:17:34.527375 19477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:34.522160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:48235 (local address 127.16.217.67:46061)
0812 03:17:34.522560 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:34.522565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:34.522792 (+ 227us) server_negotiation.cc:407] Connection header received
0812 03:17:34.522964 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:34.522968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:34.523021 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:34.523106 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:34.523815 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.524446 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.525853 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.526740 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.526875 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:34.526928 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:34.527185 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 03:17:34.527269 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:17:34.527377 19502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:34.522092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:46061 (local address 127.16.217.65:48235)
0812 03:17:34.522680 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:34.522694 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:34.522852 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:34.523136 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:34.523139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:34.523155 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:34.523515 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:34.523555 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.524628 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.524634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:34.525572 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:34.525601 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.526894 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.526900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:34.527020 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:34.527045 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:17:34.527171 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 03:17:34.527244 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":8,"thread_start_us":391,"threads_started":1}
I20260812 03:17:34.527442 19095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69d02ffd128a44d8aa82519e1b88fbb8" candidate_uuid: "11391fbc35b244b68d8579773ed9026b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc520894e91946fda723dd96d0088a4c" is_pre_election: true
I20260812 03:17:34.527873 19095 raft_consensus.cc:2454] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11391fbc35b244b68d8579773ed9026b in term 0.
I20260812 03:17:34.527994 19358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69d02ffd128a44d8aa82519e1b88fbb8" candidate_uuid: "11391fbc35b244b68d8579773ed9026b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef61e13feedf4dccac963fe18f1e833e" is_pre_election: true
I20260812 03:17:34.528264 19358 raft_consensus.cc:2454] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11391fbc35b244b68d8579773ed9026b in term 0.
I20260812 03:17:34.528431 18749 leader_election.cc:302] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11391fbc35b244b68d8579773ed9026b, cc520894e91946fda723dd96d0088a4c; no voters:
I20260812 03:17:34.528766 19492 raft_consensus.cc:2781] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:17:34.528848 19492 raft_consensus.cc:482] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:17:34.528879 19492 raft_consensus.cc:3037] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:34.529999 19492 raft_consensus.cc:504] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc520894e91946fda723dd96d0088a4c" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42051 } } peers { permanent_uuid: "11391fbc35b244b68d8579773ed9026b" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40043 } } peers { permanent_uuid: "ef61e13feedf4dccac963fe18f1e833e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46061 } }
I20260812 03:17:34.530434 19492 leader_election.cc:288] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [CANDIDATE]: Term 1 election: Requested vote from peers cc520894e91946fda723dd96d0088a4c (127.16.217.66:42051), ef61e13feedf4dccac963fe18f1e833e (127.16.217.67:46061)
I20260812 03:17:34.530968 19095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69d02ffd128a44d8aa82519e1b88fbb8" candidate_uuid: "11391fbc35b244b68d8579773ed9026b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc520894e91946fda723dd96d0088a4c"
I20260812 03:17:34.530959 19358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69d02ffd128a44d8aa82519e1b88fbb8" candidate_uuid: "11391fbc35b244b68d8579773ed9026b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef61e13feedf4dccac963fe18f1e833e"
I20260812 03:17:34.531104 19358 raft_consensus.cc:3037] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:34.531131 19095 raft_consensus.cc:3037] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:34.532881 19358 raft_consensus.cc:2454] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11391fbc35b244b68d8579773ed9026b in term 1.
I20260812 03:17:34.532871 19095 raft_consensus.cc:2454] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11391fbc35b244b68d8579773ed9026b in term 1.
I20260812 03:17:34.533416 18746 leader_election.cc:302] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 11391fbc35b244b68d8579773ed9026b, cc520894e91946fda723dd96d0088a4c, ef61e13feedf4dccac963fe18f1e833e; no voters:
I20260812 03:17:34.534487 19492 raft_consensus.cc:2781] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:34.534741 19492 raft_consensus.cc:686] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [term 1 LEADER]: Becoming Leader. State: Replica: 11391fbc35b244b68d8579773ed9026b, State: Running, Role: LEADER
I20260812 03:17:34.535107 19492 consensus_queue.cc:237] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc520894e91946fda723dd96d0088a4c" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42051 } } peers { permanent_uuid: "11391fbc35b244b68d8579773ed9026b" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40043 } } peers { permanent_uuid: "ef61e13feedf4dccac963fe18f1e833e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46061 } }
I20260812 03:17:34.538502 18601 catalog_manager.cc:5495] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11391fbc35b244b68d8579773ed9026b (127.16.217.65). New cstate: current_term: 1 leader_uuid: "11391fbc35b244b68d8579773ed9026b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc520894e91946fda723dd96d0088a4c" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11391fbc35b244b68d8579773ed9026b" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 40043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef61e13feedf4dccac963fe18f1e833e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46061 } health_report { overall_health: UNKNOWN } } }
W20260812 03:17:34.546715 18942 tablet.cc:2369] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:17:34.560921 19481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:34.556912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41278 (local address 127.16.217.65:40043)
0812 03:17:34.557088 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:34.557092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:34.557105 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:17:34.557177 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:34.557179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:34.557237 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:34.557360 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:17:34.557617 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.558117 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.559127 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:34.559369 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:34.559523 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:34.559917 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:17:34.560456 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:17:34.560635 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:34.560722 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 03:17:34.560759 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 03:17:34.568401 19358 raft_consensus.cc:1261] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e [term 1 FOLLOWER]: Refusing update from remote peer 11391fbc35b244b68d8579773ed9026b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:17:34.568401 19095 raft_consensus.cc:1261] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c [term 1 FOLLOWER]: Refusing update from remote peer 11391fbc35b244b68d8579773ed9026b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:17:34.569757 19492 consensus_queue.cc:1035] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc520894e91946fda723dd96d0088a4c" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 42051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:17:34.570390 19492 consensus_queue.cc:1035] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef61e13feedf4dccac963fe18f1e833e" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 46061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:17:34.587997 18597 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:17:34.588205 18597 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:17:34.591162 18597 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 396a89a4075d41c980d9494119ed4ac3: Sending DeleteTablet for 3 replicas of tablet 69d02ffd128a44d8aa82519e1b88fbb8
I20260812 03:17:34.592166 19317 tablet_service.cc:1514] Processing DeleteTablet for tablet 69d02ffd128a44d8aa82519e1b88fbb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:17:34 UTC) from {username='slave'} at 127.0.0.1:59396
I20260812 03:17:34.592162 18817 tablet_service.cc:1514] Processing DeleteTablet for tablet 69d02ffd128a44d8aa82519e1b88fbb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:17:34 UTC) from {username='slave'} at 127.0.0.1:41272
I20260812 03:17:34.592376 19055 tablet_service.cc:1514] Processing DeleteTablet for tablet 69d02ffd128a44d8aa82519e1b88fbb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:17:34 UTC) from {username='slave'} at 127.0.0.1:43924
I20260812 03:17:34.592911 19526 tablet_replica.cc:330] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c: stopping tablet replica
I20260812 03:17:34.592908 19524 tablet_replica.cc:330] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e: stopping tablet replica
I20260812 03:17:34.593281 19526 raft_consensus.cc:2229] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:17:34.593279 19524 raft_consensus.cc:2229] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:17:34.593634 19526 raft_consensus.cc:2258] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:17:34.593642 19524 raft_consensus.cc:2258] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:17:34.594105 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 18721
I20260812 03:17:34.594038 19525 tablet_replica.cc:330] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b: stopping tablet replica
I20260812 03:17:34.594551 19525 raft_consensus.cc:2229] T 69d02ffd128a44d8aa82519e1b88fbb8 P 11391fbc35b244b68d8579773ed9026b [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:17:34.598692 19524 ts_tablet_manager.cc:1905] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:17:34.602185 19524 ts_tablet_manager.cc:1918] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:17:34.602335 19524 log.cc:1196] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/69d02ffd128a44d8aa82519e1b88fbb8
I20260812 03:17:34.602664 19524 ts_tablet_manager.cc:1939] T 69d02ffd128a44d8aa82519e1b88fbb8 P ef61e13feedf4dccac963fe18f1e833e: Deleting consensus metadata
I20260812 03:17:34.603206 19526 ts_tablet_manager.cc:1905] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:17:34.604897 18564 catalog_manager.cc:4841] TS ef61e13feedf4dccac963fe18f1e833e (127.16.217.67:46061): tablet 69d02ffd128a44d8aa82519e1b88fbb8 (table test-table [id=718a5baaa89244d4bb4bfbed303e5484]) successfully deleted
I20260812 03:17:34.607003 19526 ts_tablet_manager.cc:1918] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:17:34.607410 19526 log.cc:1196] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/69d02ffd128a44d8aa82519e1b88fbb8
I20260812 03:17:34.607997 19526 ts_tablet_manager.cc:1939] T 69d02ffd128a44d8aa82519e1b88fbb8 P cc520894e91946fda723dd96d0088a4c: Deleting consensus metadata
I20260812 03:17:34.610008 18567 catalog_manager.cc:4841] TS cc520894e91946fda723dd96d0088a4c (127.16.217.66:42051): tablet 69d02ffd128a44d8aa82519e1b88fbb8 (table test-table [id=718a5baaa89244d4bb4bfbed303e5484]) successfully deleted
W20260812 03:17:34.613440 18567 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.217.65:40043 (error 108)
I20260812 03:17:34.613608 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 18947
I20260812 03:17:34.627837 19468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:34.627487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:40043 (local address 127.0.0.1:41294)
0812 03:17:34.627645 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:34.627744 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 03:17:34.628057 18567 catalog_manager.cc:4584] TS 11391fbc35b244b68d8579773ed9026b (127.16.217.65:40043): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 69d02ffd128a44d8aa82519e1b88fbb8: Network error: Client connection negotiation failed: client connection to 127.16.217.65:40043: connect: Connection refused (error 111)
I20260812 03:17:34.634476 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 19202
I20260812 03:17:34.647264 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 18541
2026-08-12T03:17:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1363 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:17:34 dist-test-slave-7nm7 krb5kdc[19535](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:17:34 dist-test-slave-7nm7 krb5kdc[19535](info): set up 2 sockets
Aug 12 03:17:34 dist-test-slave-7nm7 krb5kdc[19535](info): commencing operation
krb5kdc: starting...
W20260812 03:17:35.968938 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:17:36 dist-test-slave-7nm7 krb5kdc[19535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:17:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:17:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:17:36.107574 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:46773
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:42971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:46773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:36.238566 19556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:36.238876 19556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:36.238921 19556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:36.243283 19556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:36.243345 19556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:36.243366 19556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:36.243383 19556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:36.243399 19556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:36.248797 19556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:42971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:46773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:46773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:36.249840 19556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:36.250836 19556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:36.259645 19564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:36.259631 19568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:36.259704 19556 server_base.cc:1034] running on GCE node
W20260812 03:17:36.259629 19573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:36.260627 19556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:36.261686 19556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:36.262852 19556 hybrid_clock.cc:648] HybridClock initialized: now 1786504656262830 us; error 33 us; skew 500 ppm
Aug 12 03:17:36 dist-test-slave-7nm7 krb5kdc[19535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504656, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:36.265765 19556 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:17:36.266865 19556 webserver.cc:460] Webserver started at http://127.16.217.126:36513/ using document root <none> and password file <none>
I20260812 03:17:36.267442 19556 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:36.267504 19556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:36.267679 19556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:36.269786 19556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "0ad294fc32b64b27a43f2ec94958886f"
format_stamp: "Formatted at 2026-08-12 03:17:36 on dist-test-slave-7nm7"
server_key: "a910d0d47092f4b7ea65911678c1ccbe"
server_key_iv: "c98c3788e7823b26a393678b9b15d1d9"
server_key_version: "encryptionkey@0"
I20260812 03:17:36.270447 19556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "0ad294fc32b64b27a43f2ec94958886f"
format_stamp: "Formatted at 2026-08-12 03:17:36 on dist-test-slave-7nm7"
server_key: "a910d0d47092f4b7ea65911678c1ccbe"
server_key_iv: "c98c3788e7823b26a393678b9b15d1d9"
server_key_version: "encryptionkey@0"
I20260812 03:17:36.274513 19556 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:17:36.277197 19583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:36.278548 19556 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:17:36.278697 19556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "0ad294fc32b64b27a43f2ec94958886f"
format_stamp: "Formatted at 2026-08-12 03:17:36 on dist-test-slave-7nm7"
server_key: "a910d0d47092f4b7ea65911678c1ccbe"
server_key_iv: "c98c3788e7823b26a393678b9b15d1d9"
server_key_version: "encryptionkey@0"
I20260812 03:17:36.278810 19556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:36.328991 19556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:36.332590 19556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:36.332815 19556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:36.340826 19556 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:46773
I20260812 03:17:36.341063 19687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:46773 every 8 connection(s)
I20260812 03:17:36.341892 19556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:17:36.344893 19689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:36.347064 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 19556
I20260812 03:17:36.347214 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:17:36.347529 17253 external_mini_cluster.cc:1428] Setting key 833afafe5ab8de9dc04fbb3c52ebe694
I20260812 03:17:36.352159 19689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f: Bootstrap starting.
I20260812 03:17:36.354513 19689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:36.355247 19689 log.cc:824] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:17:36 dist-test-slave-7nm7 krb5kdc[19535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:17:36.357215 19689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f: No bootstrap required, opened a new log
I20260812 03:17:36.360746 19689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad294fc32b64b27a43f2ec94958886f" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46773 } }
I20260812 03:17:36.361018 19689 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:36.361112 19689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad294fc32b64b27a43f2ec94958886f, State: Initialized, Role: FOLLOWER
I20260812 03:17:36.361562 19689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad294fc32b64b27a43f2ec94958886f" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46773 } }
I20260812 03:17:36.361729 19693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:36.348924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34770 (local address 127.16.217.126:46773)
0812 03:17:36.349345 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:36.349361 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:36.349397 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:17:36.350066 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:36.350093 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:36.350494 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:36.350832 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:36.351637 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:36.352654 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:36.353523 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:36.353992 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:36.354182 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:36.356792 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:36.356797 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:36.356811 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:36.356838 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:36.358656 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:36.359122 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:36.359127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:36.359132 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:36.359246 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:36.359538 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:36.359543 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:36.359544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:36.359950 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:36.360176 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:36.360470 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 03:17:36.360769 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":113,"threads_started":1}
I20260812 03:17:36.361727 19689 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:17:36.361903 19689 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:17:36.362021 19689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:36.363718 19689 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad294fc32b64b27a43f2ec94958886f" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46773 } }
I20260812 03:17:36.364300 19689 leader_election.cc:302] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ad294fc32b64b27a43f2ec94958886f; no voters:
I20260812 03:17:36.364743 19689 leader_election.cc:288] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:17:36.364882 19696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:36.365235 19696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f [term 1 LEADER]: Becoming Leader. State: Replica: 0ad294fc32b64b27a43f2ec94958886f, State: Running, Role: LEADER
I20260812 03:17:36.365603 19696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad294fc32b64b27a43f2ec94958886f" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46773 } }
I20260812 03:17:36.366082 19689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:17:36.367395 19698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ad294fc32b64b27a43f2ec94958886f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad294fc32b64b27a43f2ec94958886f" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46773 } } }
I20260812 03:17:36.367448 19700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ad294fc32b64b27a43f2ec94958886f. Latest consensus state: current_term: 1 leader_uuid: "0ad294fc32b64b27a43f2ec94958886f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad294fc32b64b27a43f2ec94958886f" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46773 } } }
I20260812 03:17:36.367532 19698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:36.367550 19700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:36.367924 19716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:17:36.371459 19716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:17:36.378324 19716 catalog_manager.cc:1347] Generated new cluster ID: 090a9e7075394bccbd46441a09c4297e
I20260812 03:17:36.378393 19716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:17:36.392362 19716 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:17:36.393666 19716 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:17:36.402371 19716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f: Generated new TSK 0
I20260812 03:17:36.403234 19716 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
Entry for principal kudu/127.16.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab.
I20260812 03:17:36.434186 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:46773
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:42971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:36.577669 19734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:36.577983 19734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:36.578068 19734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:36.582188 19734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:36.582314 19734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:36.582417 19734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:17:36.588551 19734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:42971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.65.keytab
--principal=kudu/127.16.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:46773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:36.590374 19734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:36.591519 19734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:36.601935 19748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:36.601949 19742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:36.601949 19743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:36.602023 19734 server_base.cc:1034] running on GCE node
I20260812 03:17:36.602635 19734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:36.603267 19734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:36.604439 19734 hybrid_clock.cc:648] HybridClock initialized: now 1786504656604426 us; error 25 us; skew 500 ppm
Aug 12 03:17:36 dist-test-slave-7nm7 krb5kdc[19535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504656, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:36.607414 19734 init.cc:377] Logged in from keytab as kudu/127.16.217.65@KRBTEST.COM (short username kudu)
I20260812 03:17:36.608371 19734 webserver.cc:460] Webserver started at http://127.16.217.65:36187/ using document root <none> and password file <none>
I20260812 03:17:36.608949 19734 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:36.609012 19734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:36.609241 19734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:36.611423 19734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "7ba0607dbda24c77a42d7438980372fe"
format_stamp: "Formatted at 2026-08-12 03:17:36 on dist-test-slave-7nm7"
server_key: "6f609ae0475e8fe559f5032a33da32db"
server_key_iv: "f92eeed5627359eebd8255442fc6688d"
server_key_version: "encryptionkey@0"
I20260812 03:17:36.612021 19734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "7ba0607dbda24c77a42d7438980372fe"
format_stamp: "Formatted at 2026-08-12 03:17:36 on dist-test-slave-7nm7"
server_key: "6f609ae0475e8fe559f5032a33da32db"
server_key_iv: "f92eeed5627359eebd8255442fc6688d"
server_key_version: "encryptionkey@0"
I20260812 03:17:36.616024 19734 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 03:17:36.618676 19761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:36.620133 19734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:17:36.620326 19734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "7ba0607dbda24c77a42d7438980372fe"
format_stamp: "Formatted at 2026-08-12 03:17:36 on dist-test-slave-7nm7"
server_key: "6f609ae0475e8fe559f5032a33da32db"
server_key_iv: "f92eeed5627359eebd8255442fc6688d"
server_key_version: "encryptionkey@0"
I20260812 03:17:36.620465 19734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:36.634816 19734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:36.638890 19734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:36.639127 19734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:36.639766 19734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:36.640693 19734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:36.640766 19734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:36.640843 19734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:36.640898 19734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:36.651180 19734 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:45335
I20260812 03:17:36.651197 19981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:45335 every 8 connection(s)
I20260812 03:17:36.652128 19734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:17:36.656829 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 19734
I20260812 03:17:36.656977 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:17:36.657373 17253 external_mini_cluster.cc:1428] Setting key 454ab0ca6d74a5cf73df290019f018f1
Aug 12 03:17:36 dist-test-slave-7nm7 krb5kdc[19535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504656, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.65@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:17:36.668833 19693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:36.654005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:51761 (local address 127.16.217.126:46773)
0812 03:17:36.654154 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:36.654158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:36.655079 (+ 921us) server_negotiation.cc:407] Connection header received
0812 03:17:36.656414 (+ 1335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:36.656417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:36.656465 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:36.656542 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:36.658216 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:36.659012 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:36.660750 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:36.660973 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:36.661155 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:36.664080 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:36.664088 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:36.664093 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:36.664129 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:36.665979 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:36.666787 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:36.666792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:36.666796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:36.666899 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:36.667374 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:36.667380 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:36.667382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:36.667648 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:36.667767 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:36.668425 (+ 658us) server_negotiation.cc:299] Negotiation successful
0812 03:17:36.668585 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
WARNING: no policy specified for kudu/127.16.217.66@KRBTEST.COM; defaulting to no policy
I20260812 03:17:36.669879 19986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:36.654408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:46773 (local address 127.16.217.65:51761)
0812 03:17:36.654920 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:36.654962 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:36.656186 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:36.656815 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:36.656826 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:36.657357 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:36.658030 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:36.658050 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:36.659148 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:36.659151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:36.660612 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:36.660627 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:36.661227 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:36.661233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:36.661421 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:36.662068 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:17:36.662086 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:36.663878 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:36.666135 (+ 2257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:36.666141 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:36.666154 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:36.666653 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:36.667009 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:36.667012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:36.667014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:36.667235 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:36.667767 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:36.667773 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:36.668127 (+ 354us) client_negotiation.cc:769] Sending connection context
0812 03:17:36.668309 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 03:17:36.668709 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":127,"threads_started":1}
I20260812 03:17:36.671483 19983 heartbeater.cc:347] Connected to a master server at 127.16.217.126:46773
I20260812 03:17:36.671823 19983 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.66@KRBTEST.COM" created.
I20260812 03:17:36.672487 19983 heartbeater.cc:511] Master 127.16.217.126:46773 requested a full tablet report, sending...
I20260812 03:17:36.674386 19621 ts_manager.cc:194] Registered new tserver with Master: 7ba0607dbda24c77a42d7438980372fe (127.16.217.65:45335)
I20260812 03:17:36.676198 19621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.65@KRBTEST.COM'} at 127.16.217.65:51761
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
Entry for principal kudu/127.16.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab.
I20260812 03:17:36.693825 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:46773
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:42971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:36.810840 19992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:36.811125 19992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:36.811172 19992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:36.815310 19992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:36.815382 19992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:36.815464 19992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:17:36.820935 19992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:42971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.66.keytab
--principal=kudu/127.16.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:46773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:36.822083 19992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:36.823173 19992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:36.833093 20001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:36.833184 19992 server_base.cc:1034] running on GCE node
W20260812 03:17:36.833190 20000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:36.833343 20007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:36.833791 19992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:36.834501 19992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:36.835719 19992 hybrid_clock.cc:648] HybridClock initialized: now 1786504656835704 us; error 27 us; skew 500 ppm
Aug 12 03:17:36 dist-test-slave-7nm7 krb5kdc[19535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504656, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:36.838666 19992 init.cc:377] Logged in from keytab as kudu/127.16.217.66@KRBTEST.COM (short username kudu)
I20260812 03:17:36.839505 19992 webserver.cc:460] Webserver started at http://127.16.217.66:45173/ using document root <none> and password file <none>
I20260812 03:17:36.840034 19992 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:36.840095 19992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:36.840258 19992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:36.842368 19992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "1709b2f8016548a7bb8c34b8d2b1bad7"
format_stamp: "Formatted at 2026-08-12 03:17:36 on dist-test-slave-7nm7"
server_key: "669810a7fc66d8afe64447240773aabb"
server_key_iv: "bcb0da6d14b154c35dd9c7032431022b"
server_key_version: "encryptionkey@0"
I20260812 03:17:36.842993 19992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "1709b2f8016548a7bb8c34b8d2b1bad7"
format_stamp: "Formatted at 2026-08-12 03:17:36 on dist-test-slave-7nm7"
server_key: "669810a7fc66d8afe64447240773aabb"
server_key_iv: "bcb0da6d14b154c35dd9c7032431022b"
server_key_version: "encryptionkey@0"
I20260812 03:17:36.846936 19992 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:17:36.849557 20023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:36.850996 19992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:17:36.851110 19992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "1709b2f8016548a7bb8c34b8d2b1bad7"
format_stamp: "Formatted at 2026-08-12 03:17:36 on dist-test-slave-7nm7"
server_key: "669810a7fc66d8afe64447240773aabb"
server_key_iv: "bcb0da6d14b154c35dd9c7032431022b"
server_key_version: "encryptionkey@0"
I20260812 03:17:36.851264 19992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:36.875317 19992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:36.878983 19992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:36.879185 19992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:36.879801 19992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:36.880769 19992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:36.880827 19992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:36.880872 19992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:36.880888 19992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:36.892688 19992 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:39009
I20260812 03:17:36.892705 20255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:39009 every 8 connection(s)
I20260812 03:17:36.893715 19992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:17:36.902637 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 19992
I20260812 03:17:36.902765 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:17:36.903131 17253 external_mini_cluster.cc:1428] Setting key 4cb23a8dd64cf285cc6e6d0e2d598091
Aug 12 03:17:36 dist-test-slave-7nm7 krb5kdc[19535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504656, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.66@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:17:36.912503 19693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:36.896509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:40827 (local address 127.16.217.126:46773)
0812 03:17:36.896797 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:36.896803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:36.897717 (+ 914us) server_negotiation.cc:407] Connection header received
0812 03:17:36.899554 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:36.899558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:36.899623 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:36.899712 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:36.901240 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:36.901835 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:36.903361 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:36.903637 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:36.903778 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:36.907125 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:36.907130 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:36.907133 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:36.907186 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:36.909184 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:36.910064 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:36.910070 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:36.910072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:36.910353 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:36.910926 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:36.910931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:36.910932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:36.911203 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:36.911416 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:36.912247 (+ 831us) server_negotiation.cc:299] Negotiation successful
0812 03:17:36.912369 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260812 03:17:36.913789 20259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:36.896717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:46773 (local address 127.16.217.66:40827)
0812 03:17:36.897428 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:36.897524 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:36.899278 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:36.899927 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:36.899934 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:36.900500 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:36.901022 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:36.901037 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:36.901988 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:36.901990 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:36.903140 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:36.903151 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:36.903826 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:36.903831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:36.904097 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:36.904808 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:17:36.904825 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:36.906628 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:36.909340 (+ 2712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:36.909350 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:36.909373 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:36.909825 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:36.910526 (+ 701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:36.910530 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:36.910532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:36.910746 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:36.911435 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:36.911445 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:36.911870 (+ 425us) client_negotiation.cc:769] Sending connection context
0812 03:17:36.912131 (+ 261us) client_negotiation.cc:240] Negotiation successful
0812 03:17:36.912548 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":138,"threads_started":1}
WARNING: no policy specified for kudu/127.16.217.67@KRBTEST.COM; defaulting to no policy
I20260812 03:17:36.915207 20256 heartbeater.cc:347] Connected to a master server at 127.16.217.126:46773
I20260812 03:17:36.915717 20256 heartbeater.cc:464] Registering TS with master...
I20260812 03:17:36.916522 20256 heartbeater.cc:511] Master 127.16.217.126:46773 requested a full tablet report, sending...
I20260812 03:17:36.918304 19621 ts_manager.cc:194] Registered new tserver with Master: 1709b2f8016548a7bb8c34b8d2b1bad7 (127.16.217.66:39009)
I20260812 03:17:36.919013 19621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.66@KRBTEST.COM'} at 127.16.217.66:40827
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
Entry for principal kudu/127.16.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab.
I20260812 03:17:36.941818 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:46773
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:42971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:37.075326 20266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:37.075738 20266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:37.075843 20266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:37.079934 20266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:37.080014 20266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:37.080102 20266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:17:37.085397 20266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:42971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.67.keytab
--principal=kudu/127.16.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:46773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:37.086673 20266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:37.087733 20266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:37.097877 20275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:37.097884 20278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:37.097956 20266 server_base.cc:1034] running on GCE node
W20260812 03:17:37.097904 20276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:37.098595 20266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:37.099247 20266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:37.100411 20266 hybrid_clock.cc:648] HybridClock initialized: now 1786504657100391 us; error 31 us; skew 500 ppm
Aug 12 03:17:37 dist-test-slave-7nm7 krb5kdc[19535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504657, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:37.108117 20266 init.cc:377] Logged in from keytab as kudu/127.16.217.67@KRBTEST.COM (short username kudu)
I20260812 03:17:37.109091 20266 webserver.cc:460] Webserver started at http://127.16.217.67:46355/ using document root <none> and password file <none>
I20260812 03:17:37.109714 20266 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:37.109812 20266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:37.110071 20266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:37.112280 20266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "c7d5d2b3243e41e6a0d8110e04cb9c41"
format_stamp: "Formatted at 2026-08-12 03:17:37 on dist-test-slave-7nm7"
server_key: "27daf66b54f0e620e1c14e1f3961cda4"
server_key_iv: "9a442ecf055d62ebcd602cf700c042b0"
server_key_version: "encryptionkey@0"
I20260812 03:17:37.112886 20266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "c7d5d2b3243e41e6a0d8110e04cb9c41"
format_stamp: "Formatted at 2026-08-12 03:17:37 on dist-test-slave-7nm7"
server_key: "27daf66b54f0e620e1c14e1f3961cda4"
server_key_iv: "9a442ecf055d62ebcd602cf700c042b0"
server_key_version: "encryptionkey@0"
I20260812 03:17:37.117057 20266 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:17:37.119711 20298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:37.121096 20266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:17:37.121248 20266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "c7d5d2b3243e41e6a0d8110e04cb9c41"
format_stamp: "Formatted at 2026-08-12 03:17:37 on dist-test-slave-7nm7"
server_key: "27daf66b54f0e620e1c14e1f3961cda4"
server_key_iv: "9a442ecf055d62ebcd602cf700c042b0"
server_key_version: "encryptionkey@0"
I20260812 03:17:37.121371 20266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:37.141776 20266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:37.145594 20266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:37.145864 20266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:37.146574 20266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:37.147466 20266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:37.147541 20266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:37.147619 20266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:37.147663 20266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:37.158332 20266 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:40343
I20260812 03:17:37.158356 20534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:40343 every 8 connection(s)
I20260812 03:17:37.159279 20266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:17:37.161381 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 20266
I20260812 03:17:37.161520 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:17:37.161855 17253 external_mini_cluster.cc:1428] Setting key 0df0dc417edacc0acbeb6435134be78e
Aug 12 03:17:37 dist-test-slave-7nm7 krb5kdc[19535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504657, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.67@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:17:37.174203 19693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:37.161379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:50391 (local address 127.16.217.126:46773)
0812 03:17:37.161542 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:37.161549 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:37.162641 (+ 1092us) server_negotiation.cc:407] Connection header received
0812 03:17:37.164194 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:37.164198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:37.164253 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:37.164331 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:37.165500 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.166055 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.167406 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.167695 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.167839 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:37.170557 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:37.170560 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:37.170562 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:37.170585 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:37.172053 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:37.172603 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:37.172611 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:37.172613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:37.172685 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:37.173061 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:37.173068 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:37.173069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:37.173246 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:37.173360 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:37.173870 (+ 510us) server_negotiation.cc:299] Negotiation successful
0812 03:17:37.174022 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 03:17:37.175256 20542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:37.161827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:46773 (local address 127.16.217.67:50391)
0812 03:17:37.162486 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:37.162542 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:37.163994 (+ 1452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:37.164500 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:37.164506 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:37.164875 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:37.165357 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:37.165370 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.166194 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.166198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:37.167281 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:37.167290 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.167898 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.167902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:37.168128 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:37.168756 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:17:37.168773 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:37.170376 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:37.172192 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:37.172198 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:37.172213 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:37.172479 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:37.172779 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:17:37.172783 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:17:37.172785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:17:37.172945 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:17:37.173358 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:37.173365 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:37.173623 (+ 258us) client_negotiation.cc:769] Sending connection context
0812 03:17:37.173811 (+ 188us) client_negotiation.cc:240] Negotiation successful
0812 03:17:37.174153 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":166,"threads_started":1}
I20260812 03:17:37.176443 20538 heartbeater.cc:347] Connected to a master server at 127.16.217.126:46773
I20260812 03:17:37.176751 20538 heartbeater.cc:464] Registering TS with master...
I20260812 03:17:37.177453 20538 heartbeater.cc:511] Master 127.16.217.126:46773 requested a full tablet report, sending...
I20260812 03:17:37.178642 19621 ts_manager.cc:194] Registered new tserver with Master: c7d5d2b3243e41e6a0d8110e04cb9c41 (127.16.217.67:40343)
I20260812 03:17:37.179401 19621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.217.67@KRBTEST.COM'} at 127.16.217.67:50391
I20260812 03:17:37.188215 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:17:37.198112 19693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:37.190592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34780 (local address 127.16.217.126:46773)
0812 03:17:37.190735 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:37.190738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:37.190943 (+ 205us) server_negotiation.cc:407] Connection header received
0812 03:17:37.191070 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:37.191072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:37.191123 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:37.191200 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:37.191939 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.192463 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.193304 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.193517 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.193591 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:37.194940 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:37.194944 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:37.194946 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:37.194968 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:37.196440 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:37.196878 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:37.196881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:37.196882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:37.196956 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:37.197207 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:37.197209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:37.197210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:37.197424 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:37.197535 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:37.197858 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 03:17:37.197985 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:17:37.201221 19621 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:17:37.204697 19621 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:17:37.220075 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:37.213533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37376 (local address 127.16.217.67:40343)
0812 03:17:37.214144 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:37.214157 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:37.214180 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:17:37.214496 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:37.214515 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:37.214745 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:37.214915 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:37.215586 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.216395 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.218046 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.218795 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.218916 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:37.219254 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:37.219603 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 03:17:37.219758 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":139,"threads_started":1}
I20260812 03:17:37.220100 20557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:37.213071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:40343 (local address 127.0.0.1:37376)
0812 03:17:37.213463 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:37.213524 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:37.213653 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:37.215080 (+ 1427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:37.215086 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:37.215126 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:37.215458 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:37.215469 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.216539 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.216543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:37.217897 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:37.217911 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.218929 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.218933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:37.219039 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:37.219281 (+ 242us) client_negotiation.cc:769] Sending connection context
0812 03:17:37.219550 (+ 269us) client_negotiation.cc:240] Negotiation successful
0812 03:17:37.219709 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":58,"spinlock_wait_cycles":8704,"thread_start_us":96,"threads_started":1}
I20260812 03:17:37.220392 20570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:37.214136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45598 (local address 127.16.217.66:39009)
0812 03:17:37.215004 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:37.215017 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:37.215038 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:17:37.215111 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:37.215115 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:37.215306 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:37.215453 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:37.215839 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.216921 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.218468 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.219431 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.219538 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:37.219665 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:37.220138 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 03:17:37.220259 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":765,"thread_start_us":135,"threads_started":1}
I20260812 03:17:37.220057 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:37.213284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:45335 (local address 127.0.0.1:40538)
0812 03:17:37.213784 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:37.213798 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:37.213893 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:37.215079 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:37.215086 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:37.215124 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:37.215400 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:37.215412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.216808 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.216811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:37.218048 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:37.218056 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.219314 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.219318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:37.219419 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:37.219447 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 03:17:37.219508 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 03:17:37.219683 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":116,"spinlock_wait_cycles":5120,"thread_start_us":127,"threads_started":1}
I20260812 03:17:37.221239 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:37.213950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40538 (local address 127.16.217.65:45335)
0812 03:17:37.214607 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:37.214613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:37.214637 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:17:37.214711 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:37.214718 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:37.214878 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:37.215199 (+ 321us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:37.215492 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.216688 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.218158 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.219025 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.219852 (+ 827us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:37.220921 (+ 1069us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:37.220975 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 03:17:37.221085 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":111,"threads_started":1}
I20260812 03:17:37.221568 20567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:37.213986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:39009 (local address 127.0.0.1:45598)
0812 03:17:37.214478 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:37.214494 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:37.214579 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:37.215615 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:37.215618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:37.215639 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:37.215709 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:37.215714 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.217046 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.217050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:37.218335 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:37.218347 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.219729 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.219733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:37.219837 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:37.219859 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:17:37.219930 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 03:17:37.220005 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":76,"thread_start_us":101,"threads_started":1}
I20260812 03:17:37.223666 19860 tablet_service.cc:1467] Processing CreateTablet for tablet eb4e2d6cd3b546489d757fce59108c6b (DEFAULT_TABLE table=test-table [id=47c6926c46ee4675947b24591b33e188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:17:37.223727 20388 tablet_service.cc:1467] Processing CreateTablet for tablet eb4e2d6cd3b546489d757fce59108c6b (DEFAULT_TABLE table=test-table [id=47c6926c46ee4675947b24591b33e188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:17:37.224457 20111 tablet_service.cc:1467] Processing CreateTablet for tablet eb4e2d6cd3b546489d757fce59108c6b (DEFAULT_TABLE table=test-table [id=47c6926c46ee4675947b24591b33e188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:17:37.224977 19860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb4e2d6cd3b546489d757fce59108c6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:37.225502 20111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb4e2d6cd3b546489d757fce59108c6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:37.225601 20388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb4e2d6cd3b546489d757fce59108c6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:37.231786 20588 tablet_bootstrap.cc:492] T eb4e2d6cd3b546489d757fce59108c6b P 7ba0607dbda24c77a42d7438980372fe: Bootstrap starting.
I20260812 03:17:37.233542 20589 tablet_bootstrap.cc:492] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7: Bootstrap starting.
I20260812 03:17:37.234370 20588 tablet_bootstrap.cc:654] T eb4e2d6cd3b546489d757fce59108c6b P 7ba0607dbda24c77a42d7438980372fe: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:37.234622 20595 tablet_bootstrap.cc:492] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41: Bootstrap starting.
I20260812 03:17:37.235327 20588 log.cc:824] T eb4e2d6cd3b546489d757fce59108c6b P 7ba0607dbda24c77a42d7438980372fe: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:37.235743 20589 tablet_bootstrap.cc:654] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:37.236819 20595 tablet_bootstrap.cc:654] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:37.236974 20589 log.cc:824] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:37.237491 20588 tablet_bootstrap.cc:492] T eb4e2d6cd3b546489d757fce59108c6b P 7ba0607dbda24c77a42d7438980372fe: No bootstrap required, opened a new log
I20260812 03:17:37.237717 20588 ts_tablet_manager.cc:1397] T eb4e2d6cd3b546489d757fce59108c6b P 7ba0607dbda24c77a42d7438980372fe: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 03:17:37.238090 20595 log.cc:824] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:37.239108 20589 tablet_bootstrap.cc:492] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7: No bootstrap required, opened a new log
I20260812 03:17:37.239297 20589 ts_tablet_manager.cc:1397] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 03:17:37.240129 20595 tablet_bootstrap.cc:492] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41: No bootstrap required, opened a new log
I20260812 03:17:37.240322 20595 ts_tablet_manager.cc:1397] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:17:37.241822 20588 raft_consensus.cc:348] T eb4e2d6cd3b546489d757fce59108c6b P 7ba0607dbda24c77a42d7438980372fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7d5d2b3243e41e6a0d8110e04cb9c41" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40343 } } peers { permanent_uuid: "7ba0607dbda24c77a42d7438980372fe" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45335 } } peers { permanent_uuid: "1709b2f8016548a7bb8c34b8d2b1bad7" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39009 } }
I20260812 03:17:37.242080 20588 raft_consensus.cc:374] T eb4e2d6cd3b546489d757fce59108c6b P 7ba0607dbda24c77a42d7438980372fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:37.242149 20588 raft_consensus.cc:729] T eb4e2d6cd3b546489d757fce59108c6b P 7ba0607dbda24c77a42d7438980372fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba0607dbda24c77a42d7438980372fe, State: Initialized, Role: FOLLOWER
I20260812 03:17:37.242650 20588 consensus_queue.cc:260] T eb4e2d6cd3b546489d757fce59108c6b P 7ba0607dbda24c77a42d7438980372fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7d5d2b3243e41e6a0d8110e04cb9c41" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40343 } } peers { permanent_uuid: "7ba0607dbda24c77a42d7438980372fe" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45335 } } peers { permanent_uuid: "1709b2f8016548a7bb8c34b8d2b1bad7" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39009 } }
I20260812 03:17:37.243660 20588 ts_tablet_manager.cc:1428] T eb4e2d6cd3b546489d757fce59108c6b P 7ba0607dbda24c77a42d7438980372fe: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 03:17:37.243546 20589 raft_consensus.cc:348] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7d5d2b3243e41e6a0d8110e04cb9c41" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40343 } } peers { permanent_uuid: "7ba0607dbda24c77a42d7438980372fe" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45335 } } peers { permanent_uuid: "1709b2f8016548a7bb8c34b8d2b1bad7" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39009 } }
I20260812 03:17:37.243855 20589 raft_consensus.cc:374] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:37.243927 20589 raft_consensus.cc:729] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1709b2f8016548a7bb8c34b8d2b1bad7, State: Initialized, Role: FOLLOWER
I20260812 03:17:37.244066 19983 heartbeater.cc:503] Master 127.16.217.126:46773 was elected leader, sending a full tablet report...
I20260812 03:17:37.244400 20589 consensus_queue.cc:260] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7d5d2b3243e41e6a0d8110e04cb9c41" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40343 } } peers { permanent_uuid: "7ba0607dbda24c77a42d7438980372fe" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45335 } } peers { permanent_uuid: "1709b2f8016548a7bb8c34b8d2b1bad7" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39009 } }
I20260812 03:17:37.244529 20595 raft_consensus.cc:348] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7d5d2b3243e41e6a0d8110e04cb9c41" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40343 } } peers { permanent_uuid: "7ba0607dbda24c77a42d7438980372fe" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45335 } } peers { permanent_uuid: "1709b2f8016548a7bb8c34b8d2b1bad7" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39009 } }
I20260812 03:17:37.244803 20595 raft_consensus.cc:374] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:37.244855 20595 raft_consensus.cc:729] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7d5d2b3243e41e6a0d8110e04cb9c41, State: Initialized, Role: FOLLOWER
I20260812 03:17:37.245100 20256 heartbeater.cc:503] Master 127.16.217.126:46773 was elected leader, sending a full tablet report...
I20260812 03:17:37.245273 20595 consensus_queue.cc:260] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7d5d2b3243e41e6a0d8110e04cb9c41" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40343 } } peers { permanent_uuid: "7ba0607dbda24c77a42d7438980372fe" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45335 } } peers { permanent_uuid: "1709b2f8016548a7bb8c34b8d2b1bad7" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39009 } }
I20260812 03:17:37.245406 20589 ts_tablet_manager.cc:1428] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:17:37.246115 20595 ts_tablet_manager.cc:1428] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41: Time spent starting tablet: real 0.006s user 0.001s sys 0.006s
I20260812 03:17:37.246449 20538 heartbeater.cc:503] Master 127.16.217.126:46773 was elected leader, sending a full tablet report...
W20260812 03:17:37.395380 20257 tablet.cc:2369] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:17:37.403551 19985 tablet.cc:2369] T eb4e2d6cd3b546489d757fce59108c6b P 7ba0607dbda24c77a42d7438980372fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:17:37.410615 20541 tablet.cc:2369] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:17:37.594743 20610 raft_consensus.cc:482] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:17:37.595007 20610 raft_consensus.cc:504] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7d5d2b3243e41e6a0d8110e04cb9c41" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40343 } } peers { permanent_uuid: "7ba0607dbda24c77a42d7438980372fe" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45335 } } peers { permanent_uuid: "1709b2f8016548a7bb8c34b8d2b1bad7" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39009 } }
I20260812 03:17:37.596272 20610 leader_election.cc:288] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ba0607dbda24c77a42d7438980372fe (127.16.217.65:45335), 1709b2f8016548a7bb8c34b8d2b1bad7 (127.16.217.66:39009)
I20260812 03:17:37.600517 20570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:37.596664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:45481 (local address 127.16.217.66:39009)
0812 03:17:37.596811 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:37.596818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:37.596873 (+ 55us) server_negotiation.cc:407] Connection header received
0812 03:17:37.597055 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:37.597073 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:37.597117 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:37.597208 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:37.597477 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.598138 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.599451 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.599850 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.599980 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:37.600037 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:37.600225 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 03:17:37.600380 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 03:17:37.600517 20542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:37.596551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:39009 (local address 127.16.217.67:45481)
0812 03:17:37.596807 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:37.596824 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:37.596933 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:37.597220 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:37.597223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:37.597240 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:37.597369 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:37.597375 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.598274 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.598277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:37.599322 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:37.599332 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.599976 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.599980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:37.600086 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:37.600108 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:17:37.600209 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 03:17:37.600286 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260812 03:17:37.601007 20613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:37.596613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:45335 (local address 127.16.217.67:50087)
0812 03:17:37.597014 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:37.597031 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:37.597198 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:37.597495 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:37.597498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:37.597516 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:37.597701 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:37.597709 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.598637 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.598641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:37.599752 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:37.599763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.600489 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.600493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:37.600600 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:37.600647 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 03:17:37.600772 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 03:17:37.600842 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":131,"threads_started":1}
I20260812 03:17:37.601043 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:37.596856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:50087 (local address 127.16.217.65:45335)
0812 03:17:37.596999 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:37.597004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:37.597148 (+ 144us) server_negotiation.cc:407] Connection header received
0812 03:17:37.597322 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:37.597327 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:37.597381 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:37.597471 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:37.597830 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.598512 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.599885 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.600351 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.600500 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:37.600553 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:37.600811 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 03:17:37.600903 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 03:17:37.601328 20154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4e2d6cd3b546489d757fce59108c6b" candidate_uuid: "c7d5d2b3243e41e6a0d8110e04cb9c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1709b2f8016548a7bb8c34b8d2b1bad7" is_pre_election: true
I20260812 03:17:37.601681 20154 raft_consensus.cc:2454] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7d5d2b3243e41e6a0d8110e04cb9c41 in term 0.
I20260812 03:17:37.601816 19902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4e2d6cd3b546489d757fce59108c6b" candidate_uuid: "c7d5d2b3243e41e6a0d8110e04cb9c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ba0607dbda24c77a42d7438980372fe" is_pre_election: true
I20260812 03:17:37.602120 19902 raft_consensus.cc:2454] T eb4e2d6cd3b546489d757fce59108c6b P 7ba0607dbda24c77a42d7438980372fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7d5d2b3243e41e6a0d8110e04cb9c41 in term 0.
I20260812 03:17:37.602274 20303 leader_election.cc:302] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1709b2f8016548a7bb8c34b8d2b1bad7, c7d5d2b3243e41e6a0d8110e04cb9c41; no voters:
I20260812 03:17:37.602557 20610 raft_consensus.cc:2781] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:17:37.602653 20610 raft_consensus.cc:482] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:17:37.602725 20610 raft_consensus.cc:3037] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:37.603801 20610 raft_consensus.cc:504] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7d5d2b3243e41e6a0d8110e04cb9c41" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40343 } } peers { permanent_uuid: "7ba0607dbda24c77a42d7438980372fe" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45335 } } peers { permanent_uuid: "1709b2f8016548a7bb8c34b8d2b1bad7" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39009 } }
I20260812 03:17:37.604204 20610 leader_election.cc:288] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [CANDIDATE]: Term 1 election: Requested vote from peers 7ba0607dbda24c77a42d7438980372fe (127.16.217.65:45335), 1709b2f8016548a7bb8c34b8d2b1bad7 (127.16.217.66:39009)
I20260812 03:17:37.604627 19902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4e2d6cd3b546489d757fce59108c6b" candidate_uuid: "c7d5d2b3243e41e6a0d8110e04cb9c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ba0607dbda24c77a42d7438980372fe"
I20260812 03:17:37.604631 20154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4e2d6cd3b546489d757fce59108c6b" candidate_uuid: "c7d5d2b3243e41e6a0d8110e04cb9c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1709b2f8016548a7bb8c34b8d2b1bad7"
I20260812 03:17:37.604779 20154 raft_consensus.cc:3037] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:37.604779 19902 raft_consensus.cc:3037] T eb4e2d6cd3b546489d757fce59108c6b P 7ba0607dbda24c77a42d7438980372fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:37.606109 20154 raft_consensus.cc:2454] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7d5d2b3243e41e6a0d8110e04cb9c41 in term 1.
I20260812 03:17:37.606109 19902 raft_consensus.cc:2454] T eb4e2d6cd3b546489d757fce59108c6b P 7ba0607dbda24c77a42d7438980372fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7d5d2b3243e41e6a0d8110e04cb9c41 in term 1.
I20260812 03:17:37.606468 20303 leader_election.cc:302] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1709b2f8016548a7bb8c34b8d2b1bad7, c7d5d2b3243e41e6a0d8110e04cb9c41; no voters:
I20260812 03:17:37.606675 20610 raft_consensus.cc:2781] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:37.606925 20610 raft_consensus.cc:686] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [term 1 LEADER]: Becoming Leader. State: Replica: c7d5d2b3243e41e6a0d8110e04cb9c41, State: Running, Role: LEADER
I20260812 03:17:37.607275 20610 consensus_queue.cc:237] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7d5d2b3243e41e6a0d8110e04cb9c41" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40343 } } peers { permanent_uuid: "7ba0607dbda24c77a42d7438980372fe" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45335 } } peers { permanent_uuid: "1709b2f8016548a7bb8c34b8d2b1bad7" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39009 } }
I20260812 03:17:37.610991 19624 catalog_manager.cc:5495] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7d5d2b3243e41e6a0d8110e04cb9c41 (127.16.217.67). New cstate: current_term: 1 leader_uuid: "c7d5d2b3243e41e6a0d8110e04cb9c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7d5d2b3243e41e6a0d8110e04cb9c41" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 40343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ba0607dbda24c77a42d7438980372fe" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1709b2f8016548a7bb8c34b8d2b1bad7" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39009 } health_report { overall_health: UNKNOWN } } }
I20260812 03:17:37.676056 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:37.671808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37386 (local address 127.16.217.67:40343)
0812 03:17:37.671949 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:37.671952 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:37.671967 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:17:37.672081 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:37.672083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:37.672142 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:37.672231 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:17:37.672854 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.673453 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.674491 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.674710 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.674804 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:37.675136 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:17:37.675588 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:17:37.675682 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:37.675775 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 03:17:37.675828 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:17:37.681842 19902 raft_consensus.cc:1261] T eb4e2d6cd3b546489d757fce59108c6b P 7ba0607dbda24c77a42d7438980372fe [term 1 FOLLOWER]: Refusing update from remote peer c7d5d2b3243e41e6a0d8110e04cb9c41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:17:37.681881 20154 raft_consensus.cc:1261] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7 [term 1 FOLLOWER]: Refusing update from remote peer c7d5d2b3243e41e6a0d8110e04cb9c41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:17:37.682582 20610 consensus_queue.cc:1035] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1709b2f8016548a7bb8c34b8d2b1bad7" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:17:37.682838 20615 consensus_queue.cc:1035] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ba0607dbda24c77a42d7438980372fe" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 45335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:17:37.691811 20622 mvcc.cc:205] Tried to move back new op lower bound from 7317523077857746944 to 7317523077562314752. Current Snapshot: MvccSnapshot[applied={T|T < 7317523077857746944}]
I20260812 03:17:37.699263 19602 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:17:37.699465 19602 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:17:37.701963 19602 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0ad294fc32b64b27a43f2ec94958886f: Sending DeleteTablet for 3 replicas of tablet eb4e2d6cd3b546489d757fce59108c6b
I20260812 03:17:37.702778 20388 tablet_service.cc:1514] Processing DeleteTablet for tablet eb4e2d6cd3b546489d757fce59108c6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:17:37 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:37376
I20260812 03:17:37.702922 19860 tablet_service.cc:1514] Processing DeleteTablet for tablet eb4e2d6cd3b546489d757fce59108c6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:17:37 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:40538
I20260812 03:17:37.703017 20111 tablet_service.cc:1514] Processing DeleteTablet for tablet eb4e2d6cd3b546489d757fce59108c6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:17:37 UTC) from {username='kudu', principal='kudu/127.16.217.126@KRBTEST.COM'} at 127.0.0.1:45598
I20260812 03:17:37.703323 20639 tablet_replica.cc:330] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41: stopping tablet replica
I20260812 03:17:37.703671 20639 raft_consensus.cc:2229] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:17:37.703691 20640 tablet_replica.cc:330] T eb4e2d6cd3b546489d757fce59108c6b P 7ba0607dbda24c77a42d7438980372fe: stopping tablet replica
I20260812 03:17:37.703940 20640 raft_consensus.cc:2229] T eb4e2d6cd3b546489d757fce59108c6b P 7ba0607dbda24c77a42d7438980372fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:17:37.704087 20639 raft_consensus.cc:2258] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:17:37.704173 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 19734
I20260812 03:17:37.704243 20640 raft_consensus.cc:2258] T eb4e2d6cd3b546489d757fce59108c6b P 7ba0607dbda24c77a42d7438980372fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:17:37.704566 20641 tablet_replica.cc:330] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7: stopping tablet replica
I20260812 03:17:37.704896 20641 raft_consensus.cc:2229] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:17:37.705181 20641 raft_consensus.cc:2258] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:17:37.711182 20639 ts_tablet_manager.cc:1905] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:17:37.711867 20641 ts_tablet_manager.cc:1905] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:17:37.714653 20641 ts_tablet_manager.cc:1918] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:17:37.714767 20641 log.cc:1196] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/eb4e2d6cd3b546489d757fce59108c6b
I20260812 03:17:37.714893 20639 ts_tablet_manager.cc:1918] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:17:37.715018 20639 log.cc:1196] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/eb4e2d6cd3b546489d757fce59108c6b
I20260812 03:17:37.715126 20641 ts_tablet_manager.cc:1939] T eb4e2d6cd3b546489d757fce59108c6b P 1709b2f8016548a7bb8c34b8d2b1bad7: Deleting consensus metadata
I20260812 03:17:37.715389 20639 ts_tablet_manager.cc:1939] T eb4e2d6cd3b546489d757fce59108c6b P c7d5d2b3243e41e6a0d8110e04cb9c41: Deleting consensus metadata
I20260812 03:17:37.716615 19591 catalog_manager.cc:4841] TS 1709b2f8016548a7bb8c34b8d2b1bad7 (127.16.217.66:39009): tablet eb4e2d6cd3b546489d757fce59108c6b (table test-table [id=47c6926c46ee4675947b24591b33e188]) successfully deleted
I20260812 03:17:37.716706 19590 catalog_manager.cc:4841] TS c7d5d2b3243e41e6a0d8110e04cb9c41 (127.16.217.67:40343): tablet eb4e2d6cd3b546489d757fce59108c6b (table test-table [id=47c6926c46ee4675947b24591b33e188]) successfully deleted
W20260812 03:17:37.717964 19588 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.217.65:45335 (error 108)
I20260812 03:17:37.718314 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 19992
I20260812 03:17:37.732501 20643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:37.731007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:45335 (local address 127.0.0.1:40542)
0812 03:17:37.732268 (+ 1261us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:37.732373 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:45335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1155,"thread_start_us":932,"threads_started":1}
W20260812 03:17:37.732702 19588 catalog_manager.cc:4584] TS 7ba0607dbda24c77a42d7438980372fe (127.16.217.65:45335): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eb4e2d6cd3b546489d757fce59108c6b: Network error: Client connection negotiation failed: client connection to 127.16.217.65:45335: connect: Connection refused (error 111)
I20260812 03:17:37.739004 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 20266
I20260812 03:17:37.751418 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 19556
2026-08-12T03:17:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (3105 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T03:17:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:17:37Z Disabled control of system clock
I20260812 03:17:37.797056 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:39879
--webserver_interface=127.16.217.126
--webserver_port=0
--builtin_ntp_servers=127.16.217.84:45833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:39879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:17:37.914470 20649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:37.914757 20649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:37.914795 20649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:37.919064 20649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:37.919147 20649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:37.919162 20649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:37.919180 20649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:37.919196 20649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:37.924213 20649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:45833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:39879
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:39879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:37.925398 20649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:37.926823 20649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:37.935997 20661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:37.936167 20659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:37.935997 20657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:37.936862 20649 server_base.cc:1034] running on GCE node
I20260812 03:17:37.937467 20649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:37.938683 20649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:37.939894 20649 hybrid_clock.cc:648] HybridClock initialized: now 1786504657939853 us; error 44 us; skew 500 ppm
I20260812 03:17:37.946074 20649 webserver.cc:460] Webserver started at http://127.16.217.126:36467/ using document root <none> and password file <none>
I20260812 03:17:37.946781 20649 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:37.946882 20649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:37.947145 20649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:37.949326 20649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "5cad7cf3858143cbb8624b90f164e6aa"
format_stamp: "Formatted at 2026-08-12 03:17:37 on dist-test-slave-7nm7"
server_key: "b556984311306d2dde88d03aa8319150"
server_key_iv: "1d2d7036de9a5e1f3d5c6035d6d75d1f"
server_key_version: "encryptionkey@0"
I20260812 03:17:37.949908 20649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "5cad7cf3858143cbb8624b90f164e6aa"
format_stamp: "Formatted at 2026-08-12 03:17:37 on dist-test-slave-7nm7"
server_key: "b556984311306d2dde88d03aa8319150"
server_key_iv: "1d2d7036de9a5e1f3d5c6035d6d75d1f"
server_key_version: "encryptionkey@0"
I20260812 03:17:37.953815 20649 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:17:37.956591 20674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:37.957814 20649 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:17:37.957963 20649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "5cad7cf3858143cbb8624b90f164e6aa"
format_stamp: "Formatted at 2026-08-12 03:17:37 on dist-test-slave-7nm7"
server_key: "b556984311306d2dde88d03aa8319150"
server_key_iv: "1d2d7036de9a5e1f3d5c6035d6d75d1f"
server_key_version: "encryptionkey@0"
I20260812 03:17:37.958086 20649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:37.969818 20649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:37.970639 20649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:37.970846 20649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:37.978993 20649 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:39879
I20260812 03:17:37.979090 20783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:39879 every 8 connection(s)
I20260812 03:17:37.980268 20649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:17:37.983775 20785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:37.984880 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 20649
I20260812 03:17:37.985044 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:17:37.985387 17253 external_mini_cluster.cc:1428] Setting key 9f7cb2693b1a4707f4a2fa10821bbb7a
I20260812 03:17:37.990547 20785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa: Bootstrap starting.
I20260812 03:17:37.993381 20785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:37.994184 20785 log.cc:824] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:37.995486 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:37.987320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36832 (local address 127.16.217.126:39879)
0812 03:17:37.988030 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:37.988048 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:37.988091 (+ 43us) server_negotiation.cc:407] Connection header received
0812 03:17:37.988736 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:37.988752 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:37.989134 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:37.989407 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:37.989781 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.990853 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.992042 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:37.992540 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:37.992658 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:37.993663 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:37.993669 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:37.993682 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:37.994074 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:37.994142 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:37.994163 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:37.994299 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:37.994451 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 03:17:37.994762 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":156,"threads_started":1}
I20260812 03:17:37.996368 20785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa: No bootstrap required, opened a new log
I20260812 03:17:38.001399 20785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cad7cf3858143cbb8624b90f164e6aa" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39879 } }
I20260812 03:17:38.001688 20785 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:38.001777 20785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cad7cf3858143cbb8624b90f164e6aa, State: Initialized, Role: FOLLOWER
I20260812 03:17:38.002285 20785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cad7cf3858143cbb8624b90f164e6aa" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39879 } }
I20260812 03:17:38.002460 20785 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:17:38.002553 20785 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:17:38.002658 20785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:38.003697 20785 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cad7cf3858143cbb8624b90f164e6aa" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39879 } }
I20260812 03:17:38.004074 20785 leader_election.cc:302] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cad7cf3858143cbb8624b90f164e6aa; no voters:
I20260812 03:17:38.004456 20785 leader_election.cc:288] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:17:38.004724 20790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:38.005019 20790 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa [term 1 LEADER]: Becoming Leader. State: Replica: 5cad7cf3858143cbb8624b90f164e6aa, State: Running, Role: LEADER
I20260812 03:17:38.005436 20790 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cad7cf3858143cbb8624b90f164e6aa" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39879 } }
I20260812 03:17:38.005982 20785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:17:38.007838 20793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cad7cf3858143cbb8624b90f164e6aa. Latest consensus state: current_term: 1 leader_uuid: "5cad7cf3858143cbb8624b90f164e6aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cad7cf3858143cbb8624b90f164e6aa" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39879 } } }
I20260812 03:17:38.007807 20792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cad7cf3858143cbb8624b90f164e6aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cad7cf3858143cbb8624b90f164e6aa" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 39879 } } }
I20260812 03:17:38.008001 20792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:38.008001 20793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa [sys.catalog]: This master's current role is: LEADER
W20260812 03:17:38.011067 20814 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:17:38.011143 20814 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:17:38.011232 20816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:17:38.012089 20816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:17:38.018128 20816 catalog_manager.cc:1347] Generated new cluster ID: 3c667dc66b96472883bb3c8db8eec41b
I20260812 03:17:38.018210 20816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:17:38.034006 20816 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:17:38.037868 20816 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:17:38.047838 20816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa: Generated new TSK 0
I20260812 03:17:38.049391 20816 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:17:38.064731 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.65:0
--local_ip_for_outbound_sockets=127.16.217.65
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39879
--builtin_ntp_servers=127.16.217.84:45833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:17:38.198743 20830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:38.199012 20830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:38.199052 20830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:38.203575 20830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:38.203634 20830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:38.203692 20830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.65
I20260812 03:17:38.209479 20830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:45833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.217.65
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.65
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:38.210763 20830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:38.212198 20830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:17:38.222774 20830 server_base.cc:1034] running on GCE node
W20260812 03:17:38.222703 20842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:38.222981 20840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:38.222703 20844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:38.223568 20830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:38.224252 20830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:38.225468 20830 hybrid_clock.cc:648] HybridClock initialized: now 1786504658225435 us; error 48 us; skew 500 ppm
I20260812 03:17:38.227649 20830 webserver.cc:460] Webserver started at http://127.16.217.65:35455/ using document root <none> and password file <none>
I20260812 03:17:38.228273 20830 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:38.228372 20830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:38.228619 20830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:38.230830 20830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-0/data/instance:
uuid: "6a07550a8edf430eae8cb1132e3e0b75"
format_stamp: "Formatted at 2026-08-12 03:17:38 on dist-test-slave-7nm7"
server_key: "5e2018522688baf145c9ce12172c4410"
server_key_iv: "53742b116c3df1b8bfa40d362b6b3c84"
server_key_version: "encryptionkey@0"
I20260812 03:17:38.231408 20830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance:
uuid: "6a07550a8edf430eae8cb1132e3e0b75"
format_stamp: "Formatted at 2026-08-12 03:17:38 on dist-test-slave-7nm7"
server_key: "5e2018522688baf145c9ce12172c4410"
server_key_iv: "53742b116c3df1b8bfa40d362b6b3c84"
server_key_version: "encryptionkey@0"
I20260812 03:17:38.235309 20830 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:17:38.237924 20859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:38.239255 20830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:17:38.239409 20830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-0/wal
uuid: "6a07550a8edf430eae8cb1132e3e0b75"
format_stamp: "Formatted at 2026-08-12 03:17:38 on dist-test-slave-7nm7"
server_key: "5e2018522688baf145c9ce12172c4410"
server_key_iv: "53742b116c3df1b8bfa40d362b6b3c84"
server_key_version: "encryptionkey@0"
I20260812 03:17:38.239530 20830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:38.253509 20830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:38.254189 20830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:38.254441 20830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:38.254999 20830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:38.255909 20830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:38.255955 20830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:38.255993 20830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:38.256009 20830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:38.265116 20830 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.65:42949
I20260812 03:17:38.265148 21087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.65:42949 every 8 connection(s)
I20260812 03:17:38.266024 20830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-0/data/info.pb
I20260812 03:17:38.273979 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 20830
I20260812 03:17:38.274178 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-0/wal/instance
I20260812 03:17:38.274525 17253 external_mini_cluster.cc:1428] Setting key 740a32780ca290db6fe3e4383d066e3a
I20260812 03:17:38.275454 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:38.267930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.65:48395 (local address 127.16.217.126:39879)
0812 03:17:38.268084 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:38.268088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:38.269148 (+ 1060us) server_negotiation.cc:407] Connection header received
0812 03:17:38.270669 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:38.270673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:38.270738 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:38.270843 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:38.271712 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.272253 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.273403 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.273636 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.273743 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:38.274947 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:38.274951 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:38.274954 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:38.274973 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:38.274990 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:38.274998 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:38.275085 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:38.275290 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 03:17:38.275341 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:17:38.276448 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:38.268355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39879 (local address 127.16.217.65:48395)
0812 03:17:38.268987 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:38.269026 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:38.270458 (+ 1432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:38.271011 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:38.271017 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:38.271098 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:38.271570 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:38.271582 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.272385 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.272393 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:38.273260 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:38.273270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.273800 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.273805 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:38.273962 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:38.274695 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:17:38.274697 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:38.274735 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:17:38.274796 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:38.275110 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:38.275124 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:38.275130 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:17:38.275276 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 03:17:38.275551 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":121,"threads_started":1}
I20260812 03:17:38.276575 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.66:0
--local_ip_for_outbound_sockets=127.16.217.66
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39879
--builtin_ntp_servers=127.16.217.84:45833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:17:38.277743 21088 heartbeater.cc:347] Connected to a master server at 127.16.217.126:39879
I20260812 03:17:38.277967 21088 heartbeater.cc:464] Registering TS with master...
I20260812 03:17:38.278558 21088 heartbeater.cc:511] Master 127.16.217.126:39879 requested a full tablet report, sending...
I20260812 03:17:38.280347 20711 ts_manager.cc:194] Registered new tserver with Master: 6a07550a8edf430eae8cb1132e3e0b75 (127.16.217.65:42949)
I20260812 03:17:38.281723 20711 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.217.65:48395
W20260812 03:17:38.393052 21092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:38.393354 21092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:38.393429 21092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:38.397622 21092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:38.397701 21092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:38.397756 21092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.66
I20260812 03:17:38.403015 21092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:45833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.217.66
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.66
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:38.404206 21092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:38.405752 21092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:38.416929 21105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:38.416952 21100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:38.416942 21101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:38.417186 21092 server_base.cc:1034] running on GCE node
I20260812 03:17:38.417923 21092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:38.418761 21092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:38.419973 21092 hybrid_clock.cc:648] HybridClock initialized: now 1786504658419952 us; error 39 us; skew 500 ppm
I20260812 03:17:38.422060 21092 webserver.cc:460] Webserver started at http://127.16.217.66:36827/ using document root <none> and password file <none>
I20260812 03:17:38.422765 21092 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:38.422838 21092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:38.423058 21092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:38.425565 21092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-1/data/instance:
uuid: "835a983360594ddc858cd8ed8b8ba589"
format_stamp: "Formatted at 2026-08-12 03:17:38 on dist-test-slave-7nm7"
server_key: "4e0dcf0d1d314d2f4db79d3e1dbb3e0e"
server_key_iv: "afa88212166cf53c8d4e6a396e30c829"
server_key_version: "encryptionkey@0"
I20260812 03:17:38.426295 21092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance:
uuid: "835a983360594ddc858cd8ed8b8ba589"
format_stamp: "Formatted at 2026-08-12 03:17:38 on dist-test-slave-7nm7"
server_key: "4e0dcf0d1d314d2f4db79d3e1dbb3e0e"
server_key_iv: "afa88212166cf53c8d4e6a396e30c829"
server_key_version: "encryptionkey@0"
I20260812 03:17:38.430194 21092 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260812 03:17:38.432977 21117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:38.434635 21092 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:17:38.434777 21092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-1/wal
uuid: "835a983360594ddc858cd8ed8b8ba589"
format_stamp: "Formatted at 2026-08-12 03:17:38 on dist-test-slave-7nm7"
server_key: "4e0dcf0d1d314d2f4db79d3e1dbb3e0e"
server_key_iv: "afa88212166cf53c8d4e6a396e30c829"
server_key_version: "encryptionkey@0"
I20260812 03:17:38.434991 21092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:38.451598 21092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:38.452395 21092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:38.452605 21092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:38.453251 21092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:38.454372 21092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:38.454443 21092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:38.454514 21092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:38.454564 21092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:38.464087 21092 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.66:44963
I20260812 03:17:38.464120 21340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.66:44963 every 8 connection(s)
I20260812 03:17:38.465044 21092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-1/data/info.pb
I20260812 03:17:38.475282 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:38.467125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.66:43211 (local address 127.16.217.126:39879)
0812 03:17:38.467291 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:38.467297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:38.468440 (+ 1143us) server_negotiation.cc:407] Connection header received
0812 03:17:38.469813 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:38.469816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:38.469864 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:38.469941 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:38.470939 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.471494 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.472666 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.472874 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.472956 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:38.474623 (+ 1667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:38.474626 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:38.474628 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:38.474641 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:38.474659 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:38.474667 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:38.474738 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:38.475006 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 03:17:38.475152 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 03:17:38.475031 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 21092
I20260812 03:17:38.475544 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-1/wal/instance
I20260812 03:17:38.475919 17253 external_mini_cluster.cc:1428] Setting key 6427e527371b6705679db71437911424
I20260812 03:17:38.476683 21347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:38.467574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39879 (local address 127.16.217.66:43211)
0812 03:17:38.468293 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:38.468334 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:38.469617 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:38.470114 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:38.470120 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:38.470199 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:38.470791 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:38.470805 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.471632 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.471635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:38.472542 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:38.472552 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.473011 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.473014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:38.473437 (+ 423us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:38.474120 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:17:38.474122 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:38.474168 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:17:38.474415 (+ 247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:38.474796 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:38.474807 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:38.474815 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 03:17:38.474977 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 03:17:38.475579 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":132,"threads_started":1}
I20260812 03:17:38.478056 21343 heartbeater.cc:347] Connected to a master server at 127.16.217.126:39879
I20260812 03:17:38.478339 21343 heartbeater.cc:464] Registering TS with master...
I20260812 03:17:38.478576 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.217.67:0
--local_ip_for_outbound_sockets=127.16.217.67
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39879
--builtin_ntp_servers=127.16.217.84:45833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:17:38.478966 21343 heartbeater.cc:511] Master 127.16.217.126:39879 requested a full tablet report, sending...
I20260812 03:17:38.480057 20705 ts_manager.cc:194] Registered new tserver with Master: 835a983360594ddc858cd8ed8b8ba589 (127.16.217.66:44963)
I20260812 03:17:38.480659 20705 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.217.66:43211
W20260812 03:17:38.599303 21351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:38.599552 21351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:38.599589 21351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:38.604346 21351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:38.604408 21351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:38.604486 21351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.217.67
I20260812 03:17:38.609442 21351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:45833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.217.67
--webserver_port=0
--tserver_master_addrs=127.16.217.126:39879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.217.67
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:38.610571 21351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:38.612010 21351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:38.621658 21361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:38.621722 21360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:38.621760 21351 server_base.cc:1034] running on GCE node
W20260812 03:17:38.621902 21368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:38.622360 21351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:38.622985 21351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:38.624154 21351 hybrid_clock.cc:648] HybridClock initialized: now 1786504658624126 us; error 41 us; skew 500 ppm
I20260812 03:17:38.625972 21351 webserver.cc:460] Webserver started at http://127.16.217.67:42635/ using document root <none> and password file <none>
I20260812 03:17:38.626622 21351 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:38.626688 21351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:38.626940 21351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:38.629022 21351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-2/data/instance:
uuid: "972dbc894e28409b8c8036183d23ad60"
format_stamp: "Formatted at 2026-08-12 03:17:38 on dist-test-slave-7nm7"
server_key: "aaa04be91c799e516a401d557da1958b"
server_key_iv: "6fc2c5733c291edd162a9e55b23c17ae"
server_key_version: "encryptionkey@0"
I20260812 03:17:38.629590 21351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance:
uuid: "972dbc894e28409b8c8036183d23ad60"
format_stamp: "Formatted at 2026-08-12 03:17:38 on dist-test-slave-7nm7"
server_key: "aaa04be91c799e516a401d557da1958b"
server_key_iv: "6fc2c5733c291edd162a9e55b23c17ae"
server_key_version: "encryptionkey@0"
I20260812 03:17:38.633337 21351 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 03:17:38.635926 21384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:38.637171 21351 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:17:38.637310 21351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-2/wal
uuid: "972dbc894e28409b8c8036183d23ad60"
format_stamp: "Formatted at 2026-08-12 03:17:38 on dist-test-slave-7nm7"
server_key: "aaa04be91c799e516a401d557da1958b"
server_key_iv: "6fc2c5733c291edd162a9e55b23c17ae"
server_key_version: "encryptionkey@0"
I20260812 03:17:38.637424 21351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:38.650851 21351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:38.651499 21351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:38.651707 21351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:38.652292 21351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:17:38.653254 21351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:17:38.653328 21351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:38.653401 21351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:17:38.653445 21351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:38.662289 21593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.67:42257 every 8 connection(s)
I20260812 03:17:38.662286 21351 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.67:42257
I20260812 03:17:38.663306 21351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-2/data/info.pb
I20260812 03:17:38.666523 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 21351
I20260812 03:17:38.666649 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-2/wal/instance
I20260812 03:17:38.666962 17253 external_mini_cluster.cc:1428] Setting key 808a61c33653b47b406a377f578bbfa1
I20260812 03:17:38.672958 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:38.665112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:58837 (local address 127.16.217.126:39879)
0812 03:17:38.665277 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:38.665280 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:38.666341 (+ 1061us) server_negotiation.cc:407] Connection header received
0812 03:17:38.668028 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:38.668030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:38.668076 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:38.668186 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:38.669391 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.669968 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.671187 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.671402 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.671495 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:38.672549 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:38.672553 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:38.672556 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:38.672568 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:38.672585 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:38.672592 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:38.672654 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:38.672799 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 03:17:38.672846 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 03:17:38.673825 21596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:38.665494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:39879 (local address 127.16.217.67:58837)
0812 03:17:38.666142 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:38.666182 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:38.667826 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:38.668406 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:38.668417 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:38.668527 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:17:38.669218 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:38.669237 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.670078 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.670081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:38.671027 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:38.671037 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.671548 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.671551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:38.671687 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:38.672305 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:17:38.672306 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:17:38.672344 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:17:38.672407 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:17:38.672670 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:17:38.672676 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:17:38.672681 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:17:38.672781 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 03:17:38.672991 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":124,"threads_started":1}
I20260812 03:17:38.675055 21594 heartbeater.cc:347] Connected to a master server at 127.16.217.126:39879
I20260812 03:17:38.675321 21594 heartbeater.cc:464] Registering TS with master...
I20260812 03:17:38.675894 21594 heartbeater.cc:511] Master 127.16.217.126:39879 requested a full tablet report, sending...
I20260812 03:17:38.676994 20705 ts_manager.cc:194] Registered new tserver with Master: 972dbc894e28409b8c8036183d23ad60 (127.16.217.67:42257)
I20260812 03:17:38.677603 20705 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.217.67:58837
I20260812 03:17:38.680872 17253 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:17:38.686892 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:38.682913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36836 (local address 127.16.217.126:39879)
0812 03:17:38.683041 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:38.683045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:38.683192 (+ 147us) server_negotiation.cc:407] Connection header received
0812 03:17:38.683292 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:38.683295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:38.683331 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:38.683396 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:38.683612 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.684096 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.684970 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.685314 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.685413 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:38.686212 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:38.686215 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:38.686217 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:38.686484 (+ 267us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:38.686511 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:38.686521 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:38.686583 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:38.686723 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 03:17:38.686778 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:17:38.690171 20705 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:17:38.693264 20705 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:17:38.709106 21620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:38.702757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:42949 (local address 127.0.0.1:34730)
0812 03:17:38.703205 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:38.703252 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:38.703374 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:38.704293 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:38.704301 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:38.704329 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:38.704482 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:38.704488 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.706009 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.706015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:38.707535 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:38.707546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.708532 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.708535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:38.708650 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:38.708698 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 03:17:38.708817 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 03:17:38.708953 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":116,"threads_started":1}
I20260812 03:17:38.709147 21621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:38.702886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34730 (local address 127.16.217.65:42949)
0812 03:17:38.703693 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:38.703704 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:38.703725 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:17:38.703789 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:38.703793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:38.703994 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:38.704685 (+ 691us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:38.704852 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.705903 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.707673 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.708405 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.708523 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:38.708721 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:38.708843 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 03:17:38.708948 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":134,"threads_started":1}
I20260812 03:17:38.709345 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:38.702829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:44963 (local address 127.0.0.1:35500)
0812 03:17:38.703305 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:38.703322 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:38.703421 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:38.705191 (+ 1770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:38.705194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:38.705212 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:38.705348 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:38.705357 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.706759 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.706763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:38.707935 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:38.707947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.708969 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.708972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:38.709053 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:38.709075 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:17:38.709137 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 03:17:38.709211 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
I20260812 03:17:38.709884 21622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:38.703518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.67:42257 (local address 127.0.0.1:51602)
0812 03:17:38.704730 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:38.704742 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:38.704857 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:38.705418 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:38.705421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:38.705440 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:38.705533 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:38.705540 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.706989 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.706992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:38.708162 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:38.708171 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.709416 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.709443 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:38.709545 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:38.709580 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 03:17:38.709680 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 03:17:38.709752 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1145,"mutex_wait_us":19,"thread_start_us":108,"threads_started":1}
I20260812 03:17:38.709964 21623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:38.703888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51602 (local address 127.16.217.67:42257)
0812 03:17:38.704274 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:38.704285 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:38.704840 (+ 555us) server_negotiation.cc:407] Connection header received
0812 03:17:38.704976 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:38.704980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:38.705256 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:38.705428 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:38.705639 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.706871 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.708282 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.709045 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.709154 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:38.709351 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:38.709716 (+ 365us) server_negotiation.cc:299] Negotiation successful
0812 03:17:38.709806 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":190,"threads_started":1}
I20260812 03:17:38.713091 20952 tablet_service.cc:1467] Processing CreateTablet for tablet 17c94e4e397b44dfb02776443f2e2e0f (DEFAULT_TABLE table=test-table [id=e8f8377d9d66409b94c94cf11aac96f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:17:38.713524 21479 tablet_service.cc:1467] Processing CreateTablet for tablet 17c94e4e397b44dfb02776443f2e2e0f (DEFAULT_TABLE table=test-table [id=e8f8377d9d66409b94c94cf11aac96f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:17:38.714579 20952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17c94e4e397b44dfb02776443f2e2e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:38.714727 21479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17c94e4e397b44dfb02776443f2e2e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:38.719161 21628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:38.704326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35500 (local address 127.16.217.66:44963)
0812 03:17:38.704705 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:38.704733 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:38.704756 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:17:38.704830 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:38.704836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:38.705009 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:38.705164 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:38.705491 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.706622 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.708084 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.708845 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.718320 (+ 9475us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:38.718602 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:38.718674 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 03:17:38.718801 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":147,"threads_started":1}
I20260812 03:17:38.721621 21220 tablet_service.cc:1467] Processing CreateTablet for tablet 17c94e4e397b44dfb02776443f2e2e0f (DEFAULT_TABLE table=test-table [id=e8f8377d9d66409b94c94cf11aac96f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:17:38.722455 21632 tablet_bootstrap.cc:492] T 17c94e4e397b44dfb02776443f2e2e0f P 6a07550a8edf430eae8cb1132e3e0b75: Bootstrap starting.
I20260812 03:17:38.722663 21220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17c94e4e397b44dfb02776443f2e2e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:38.722923 21630 tablet_bootstrap.cc:492] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60: Bootstrap starting.
I20260812 03:17:38.725163 21632 tablet_bootstrap.cc:654] T 17c94e4e397b44dfb02776443f2e2e0f P 6a07550a8edf430eae8cb1132e3e0b75: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:38.725173 21630 tablet_bootstrap.cc:654] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:38.725976 21632 log.cc:824] T 17c94e4e397b44dfb02776443f2e2e0f P 6a07550a8edf430eae8cb1132e3e0b75: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:38.726100 21630 log.cc:824] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:38.728310 21632 tablet_bootstrap.cc:492] T 17c94e4e397b44dfb02776443f2e2e0f P 6a07550a8edf430eae8cb1132e3e0b75: No bootstrap required, opened a new log
I20260812 03:17:38.728529 21632 ts_tablet_manager.cc:1397] T 17c94e4e397b44dfb02776443f2e2e0f P 6a07550a8edf430eae8cb1132e3e0b75: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 03:17:38.729431 21630 tablet_bootstrap.cc:492] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60: No bootstrap required, opened a new log
I20260812 03:17:38.729641 21630 ts_tablet_manager.cc:1397] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 03:17:38.730863 21639 tablet_bootstrap.cc:492] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589: Bootstrap starting.
I20260812 03:17:38.732944 21632 raft_consensus.cc:348] T 17c94e4e397b44dfb02776443f2e2e0f P 6a07550a8edf430eae8cb1132e3e0b75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a07550a8edf430eae8cb1132e3e0b75" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42949 } } peers { permanent_uuid: "835a983360594ddc858cd8ed8b8ba589" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 44963 } } peers { permanent_uuid: "972dbc894e28409b8c8036183d23ad60" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 42257 } }
I20260812 03:17:38.733256 21632 raft_consensus.cc:374] T 17c94e4e397b44dfb02776443f2e2e0f P 6a07550a8edf430eae8cb1132e3e0b75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:38.733314 21632 raft_consensus.cc:729] T 17c94e4e397b44dfb02776443f2e2e0f P 6a07550a8edf430eae8cb1132e3e0b75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a07550a8edf430eae8cb1132e3e0b75, State: Initialized, Role: FOLLOWER
I20260812 03:17:38.733503 21639 tablet_bootstrap.cc:654] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:38.733599 21630 raft_consensus.cc:348] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a07550a8edf430eae8cb1132e3e0b75" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42949 } } peers { permanent_uuid: "835a983360594ddc858cd8ed8b8ba589" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 44963 } } peers { permanent_uuid: "972dbc894e28409b8c8036183d23ad60" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 42257 } }
I20260812 03:17:38.733860 21630 raft_consensus.cc:374] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:38.733788 21632 consensus_queue.cc:260] T 17c94e4e397b44dfb02776443f2e2e0f P 6a07550a8edf430eae8cb1132e3e0b75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a07550a8edf430eae8cb1132e3e0b75" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42949 } } peers { permanent_uuid: "835a983360594ddc858cd8ed8b8ba589" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 44963 } } peers { permanent_uuid: "972dbc894e28409b8c8036183d23ad60" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 42257 } }
I20260812 03:17:38.733916 21630 raft_consensus.cc:729] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972dbc894e28409b8c8036183d23ad60, State: Initialized, Role: FOLLOWER
I20260812 03:17:38.734352 21630 consensus_queue.cc:260] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a07550a8edf430eae8cb1132e3e0b75" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42949 } } peers { permanent_uuid: "835a983360594ddc858cd8ed8b8ba589" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 44963 } } peers { permanent_uuid: "972dbc894e28409b8c8036183d23ad60" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 42257 } }
I20260812 03:17:38.734418 21639 log.cc:824] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:38.734833 21088 heartbeater.cc:503] Master 127.16.217.126:39879 was elected leader, sending a full tablet report...
I20260812 03:17:38.734938 21632 ts_tablet_manager.cc:1428] T 17c94e4e397b44dfb02776443f2e2e0f P 6a07550a8edf430eae8cb1132e3e0b75: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:17:38.735242 21630 ts_tablet_manager.cc:1428] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:17:38.735955 21594 heartbeater.cc:503] Master 127.16.217.126:39879 was elected leader, sending a full tablet report...
I20260812 03:17:38.737762 21639 tablet_bootstrap.cc:492] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589: No bootstrap required, opened a new log
I20260812 03:17:38.737987 21639 ts_tablet_manager.cc:1397] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 03:17:38.742398 21639 raft_consensus.cc:348] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a07550a8edf430eae8cb1132e3e0b75" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42949 } } peers { permanent_uuid: "835a983360594ddc858cd8ed8b8ba589" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 44963 } } peers { permanent_uuid: "972dbc894e28409b8c8036183d23ad60" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 42257 } }
I20260812 03:17:38.742699 21639 raft_consensus.cc:374] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:38.742761 21639 raft_consensus.cc:729] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 835a983360594ddc858cd8ed8b8ba589, State: Initialized, Role: FOLLOWER
I20260812 03:17:38.743286 21639 consensus_queue.cc:260] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a07550a8edf430eae8cb1132e3e0b75" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42949 } } peers { permanent_uuid: "835a983360594ddc858cd8ed8b8ba589" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 44963 } } peers { permanent_uuid: "972dbc894e28409b8c8036183d23ad60" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 42257 } }
I20260812 03:17:38.744081 21343 heartbeater.cc:503] Master 127.16.217.126:39879 was elected leader, sending a full tablet report...
I20260812 03:17:38.744369 21639 ts_tablet_manager.cc:1428] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
W20260812 03:17:38.767325 21089 tablet.cc:2369] T 17c94e4e397b44dfb02776443f2e2e0f P 6a07550a8edf430eae8cb1132e3e0b75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:17:38.887235 21642 raft_consensus.cc:482] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:17:38.887468 21642 raft_consensus.cc:504] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a07550a8edf430eae8cb1132e3e0b75" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42949 } } peers { permanent_uuid: "835a983360594ddc858cd8ed8b8ba589" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 44963 } } peers { permanent_uuid: "972dbc894e28409b8c8036183d23ad60" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 42257 } }
I20260812 03:17:38.888653 21642 leader_election.cc:288] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a07550a8edf430eae8cb1132e3e0b75 (127.16.217.65:42949), 835a983360594ddc858cd8ed8b8ba589 (127.16.217.66:44963)
I20260812 03:17:38.892740 21621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:38.889178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:46629 (local address 127.16.217.65:42949)
0812 03:17:38.889320 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:38.889324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:38.889339 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:17:38.889401 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:38.889406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:38.889461 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:38.889551 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:38.889980 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.890714 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.891806 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.892193 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.892322 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:38.892375 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:38.892552 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 03:17:38.892620 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:17:38.892740 21596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:38.888964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:42949 (local address 127.16.217.67:46629)
0812 03:17:38.889082 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:38.889099 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:38.889217 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:38.889690 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:38.889696 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:38.889716 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:38.889823 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:38.889828 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.890828 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.890833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:38.891651 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:38.891686 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.892314 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.892318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:38.892420 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:38.892443 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:17:38.892534 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 03:17:38.892602 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":33}
I20260812 03:17:38.892828 21628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:38.889072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.217.67:36631 (local address 127.16.217.66:44963)
0812 03:17:38.889221 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:38.889226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:38.889541 (+ 315us) server_negotiation.cc:407] Connection header received
0812 03:17:38.889749 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:38.889753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:38.889807 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:38.889887 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:17:38.890202 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.890822 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.891880 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.892267 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.892397 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:38.892446 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:38.892626 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 03:17:38.892696 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:17:38.892890 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:38.888964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.66:44963 (local address 127.16.217.67:36631)
0812 03:17:38.889448 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:38.889462 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:17:38.889607 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:17:38.889921 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:17:38.889925 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:17:38.889945 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:17:38.890060 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:38.890065 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.890945 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.890949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:38.891743 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:17:38.891752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.892393 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.892397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:17:38.892494 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:38.892520 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 03:17:38.892608 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 03:17:38.892682 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":154,"thread_start_us":146,"threads_started":1}
I20260812 03:17:38.893452 20995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17c94e4e397b44dfb02776443f2e2e0f" candidate_uuid: "972dbc894e28409b8c8036183d23ad60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a07550a8edf430eae8cb1132e3e0b75" is_pre_election: true
I20260812 03:17:38.893476 21259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17c94e4e397b44dfb02776443f2e2e0f" candidate_uuid: "972dbc894e28409b8c8036183d23ad60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "835a983360594ddc858cd8ed8b8ba589" is_pre_election: true
I20260812 03:17:38.893759 21259 raft_consensus.cc:2454] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972dbc894e28409b8c8036183d23ad60 in term 0.
I20260812 03:17:38.893770 20995 raft_consensus.cc:2454] T 17c94e4e397b44dfb02776443f2e2e0f P 6a07550a8edf430eae8cb1132e3e0b75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972dbc894e28409b8c8036183d23ad60 in term 0.
I20260812 03:17:38.894317 21387 leader_election.cc:302] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6a07550a8edf430eae8cb1132e3e0b75, 835a983360594ddc858cd8ed8b8ba589, 972dbc894e28409b8c8036183d23ad60; no voters:
I20260812 03:17:38.894610 21642 raft_consensus.cc:2781] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:17:38.894706 21642 raft_consensus.cc:482] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:17:38.894773 21642 raft_consensus.cc:3037] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:38.895812 21642 raft_consensus.cc:504] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a07550a8edf430eae8cb1132e3e0b75" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42949 } } peers { permanent_uuid: "835a983360594ddc858cd8ed8b8ba589" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 44963 } } peers { permanent_uuid: "972dbc894e28409b8c8036183d23ad60" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 42257 } }
I20260812 03:17:38.896220 21642 leader_election.cc:288] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [CANDIDATE]: Term 1 election: Requested vote from peers 6a07550a8edf430eae8cb1132e3e0b75 (127.16.217.65:42949), 835a983360594ddc858cd8ed8b8ba589 (127.16.217.66:44963)
I20260812 03:17:38.896533 20995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17c94e4e397b44dfb02776443f2e2e0f" candidate_uuid: "972dbc894e28409b8c8036183d23ad60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a07550a8edf430eae8cb1132e3e0b75"
I20260812 03:17:38.896595 21259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17c94e4e397b44dfb02776443f2e2e0f" candidate_uuid: "972dbc894e28409b8c8036183d23ad60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "835a983360594ddc858cd8ed8b8ba589"
I20260812 03:17:38.896687 20995 raft_consensus.cc:3037] T 17c94e4e397b44dfb02776443f2e2e0f P 6a07550a8edf430eae8cb1132e3e0b75 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:38.896723 21259 raft_consensus.cc:3037] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:38.897708 20995 raft_consensus.cc:2454] T 17c94e4e397b44dfb02776443f2e2e0f P 6a07550a8edf430eae8cb1132e3e0b75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dbc894e28409b8c8036183d23ad60 in term 1.
I20260812 03:17:38.897778 21259 raft_consensus.cc:2454] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dbc894e28409b8c8036183d23ad60 in term 1.
I20260812 03:17:38.898108 21387 leader_election.cc:302] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 835a983360594ddc858cd8ed8b8ba589, 972dbc894e28409b8c8036183d23ad60; no voters:
I20260812 03:17:38.898316 21642 raft_consensus.cc:2781] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:38.898516 21642 raft_consensus.cc:686] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [term 1 LEADER]: Becoming Leader. State: Replica: 972dbc894e28409b8c8036183d23ad60, State: Running, Role: LEADER
I20260812 03:17:38.898851 21642 consensus_queue.cc:237] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a07550a8edf430eae8cb1132e3e0b75" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42949 } } peers { permanent_uuid: "835a983360594ddc858cd8ed8b8ba589" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 44963 } } peers { permanent_uuid: "972dbc894e28409b8c8036183d23ad60" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 42257 } }
I20260812 03:17:38.902276 20705 catalog_manager.cc:5495] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 reported cstate change: term changed from 0 to 1, leader changed from <none> to 972dbc894e28409b8c8036183d23ad60 (127.16.217.67). New cstate: current_term: 1 leader_uuid: "972dbc894e28409b8c8036183d23ad60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a07550a8edf430eae8cb1132e3e0b75" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42949 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "835a983360594ddc858cd8ed8b8ba589" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 44963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dbc894e28409b8c8036183d23ad60" member_type: VOTER last_known_addr { host: "127.16.217.67" port: 42257 } health_report { overall_health: HEALTHY } } }
W20260812 03:17:38.914361 21595 tablet.cc:2369] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:17:38.946601 21623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:38.941837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51618 (local address 127.16.217.67:42257)
0812 03:17:38.942010 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:38.942014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:38.942029 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:17:38.942128 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:38.942130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:38.942191 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:38.942369 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:17:38.942603 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.943194 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.944483 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:38.944912 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:38.945063 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:38.945610 (+ 547us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:17:38.946062 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:17:38.946186 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:38.946404 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 03:17:38.946460 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:17:38.953365 21259 raft_consensus.cc:1261] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589 [term 1 FOLLOWER]: Refusing update from remote peer 972dbc894e28409b8c8036183d23ad60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:17:38.953366 20995 raft_consensus.cc:1261] T 17c94e4e397b44dfb02776443f2e2e0f P 6a07550a8edf430eae8cb1132e3e0b75 [term 1 FOLLOWER]: Refusing update from remote peer 972dbc894e28409b8c8036183d23ad60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:17:38.954161 21647 consensus_queue.cc:1035] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "835a983360594ddc858cd8ed8b8ba589" member_type: VOTER last_known_addr { host: "127.16.217.66" port: 44963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:17:38.954351 21642 consensus_queue.cc:1035] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a07550a8edf430eae8cb1132e3e0b75" member_type: VOTER last_known_addr { host: "127.16.217.65" port: 42949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:17:38.962675 21652 mvcc.cc:205] Tried to move back new op lower bound from 7317523083065556992 to 7317523082852462592. Current Snapshot: MvccSnapshot[applied={T|T < 7317523083065556992}]
I20260812 03:17:38.972482 20711 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:17:38.972698 20711 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:17:38.975628 20711 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5cad7cf3858143cbb8624b90f164e6aa: Sending DeleteTablet for 3 replicas of tablet 17c94e4e397b44dfb02776443f2e2e0f
I20260812 03:17:38.976605 21479 tablet_service.cc:1514] Processing DeleteTablet for tablet 17c94e4e397b44dfb02776443f2e2e0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:17:38 UTC) from {username='slave'} at 127.0.0.1:51602
I20260812 03:17:38.976691 20952 tablet_service.cc:1514] Processing DeleteTablet for tablet 17c94e4e397b44dfb02776443f2e2e0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:17:38 UTC) from {username='slave'} at 127.0.0.1:34730
I20260812 03:17:38.977234 21220 tablet_service.cc:1514] Processing DeleteTablet for tablet 17c94e4e397b44dfb02776443f2e2e0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:17:38 UTC) from {username='slave'} at 127.0.0.1:35500
I20260812 03:17:38.977406 21665 tablet_replica.cc:330] T 17c94e4e397b44dfb02776443f2e2e0f P 6a07550a8edf430eae8cb1132e3e0b75: stopping tablet replica
I20260812 03:17:38.977711 21670 tablet_replica.cc:330] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589: stopping tablet replica
I20260812 03:17:38.977808 21665 raft_consensus.cc:2229] T 17c94e4e397b44dfb02776443f2e2e0f P 6a07550a8edf430eae8cb1132e3e0b75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:17:38.977947 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 20830
I20260812 03:17:38.978156 21665 raft_consensus.cc:2258] T 17c94e4e397b44dfb02776443f2e2e0f P 6a07550a8edf430eae8cb1132e3e0b75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:17:38.978725 21670 raft_consensus.cc:2229] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:17:38.979079 21670 raft_consensus.cc:2258] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:17:38.988787 21670 ts_tablet_manager.cc:1905] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:17:38.992897 21670 ts_tablet_manager.cc:1918] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:17:38.993065 21670 log.cc:1196] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-1/wal/wals/17c94e4e397b44dfb02776443f2e2e0f
I20260812 03:17:38.993505 21670 ts_tablet_manager.cc:1939] T 17c94e4e397b44dfb02776443f2e2e0f P 835a983360594ddc858cd8ed8b8ba589: Deleting consensus metadata
I20260812 03:17:38.995657 20685 catalog_manager.cc:4841] TS 835a983360594ddc858cd8ed8b8ba589 (127.16.217.66:44963): tablet 17c94e4e397b44dfb02776443f2e2e0f (table test-table [id=e8f8377d9d66409b94c94cf11aac96f9]) successfully deleted
I20260812 03:17:39.002342 21664 tablet_replica.cc:330] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60: stopping tablet replica
W20260812 03:17:39.002658 20682 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.217.65:42949 (error 108)
I20260812 03:17:39.002779 21664 raft_consensus.cc:2229] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:17:39.003314 21664 raft_consensus.cc:2258] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:17:39.003619 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 21092
I20260812 03:17:39.039757 21664 ts_tablet_manager.cc:1905] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:17:39.039973 21620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:39.039470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.65:42949 (local address 127.0.0.1:34740)
0812 03:17:39.039744 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 03:17:39.039849 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.65:42949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
W20260812 03:17:39.040572 20682 catalog_manager.cc:4584] TS 6a07550a8edf430eae8cb1132e3e0b75 (127.16.217.65:42949): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 17c94e4e397b44dfb02776443f2e2e0f: Network error: Client connection negotiation failed: client connection to 127.16.217.65:42949: connect: Connection refused (error 111)
I20260812 03:17:39.042850 21664 ts_tablet_manager.cc:1918] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:17:39.042968 21664 log.cc:1196] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60: Deleting WAL directory at /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504382969880-17253-0/minicluster-data/ts-2/wal/wals/17c94e4e397b44dfb02776443f2e2e0f
I20260812 03:17:39.043324 21664 ts_tablet_manager.cc:1939] T 17c94e4e397b44dfb02776443f2e2e0f P 972dbc894e28409b8c8036183d23ad60: Deleting consensus metadata
I20260812 03:17:39.044735 20682 catalog_manager.cc:4841] TS 972dbc894e28409b8c8036183d23ad60 (127.16.217.67:42257): tablet 17c94e4e397b44dfb02776443f2e2e0f (table test-table [id=e8f8377d9d66409b94c94cf11aac96f9]) successfully deleted
I20260812 03:17:39.046132 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 21351
I20260812 03:17:39.067642 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 20649
2026-08-12T03:17:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1318 ms)
[----------] 6 tests from EncryptionPolicyTest (12934 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:17:39 dist-test-slave-7nm7 krb5kdc[21675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:17:39 dist-test-slave-7nm7 krb5kdc[21675](info): set up 2 sockets
Aug 12 03:17:39 dist-test-slave-7nm7 krb5kdc[21675](info): commencing operation
krb5kdc: starting...
W20260812 03:17:40.396849 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:17:40 dist-test-slave-7nm7 krb5kdc[21675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:17:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:17:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:17:40.529445 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37527
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:40.645709 21701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:40.646040 21701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:40.646134 21701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:40.650398 21701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:40.650524 21701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:40.650566 21701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:40.650604 21701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:40.650638 21701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:40.656456 21701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37527
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:40.657662 21701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:40.658855 21701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:40.667639 21707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:40.667685 21711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:40.667652 21706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:40.668243 21701 server_base.cc:1034] running on GCE node
I20260812 03:17:40.668875 21701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:40.670130 21701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:40.671320 21701 hybrid_clock.cc:648] HybridClock initialized: now 1786504660671302 us; error 35 us; skew 500 ppm
Aug 12 03:17:40 dist-test-slave-7nm7 krb5kdc[21675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504660, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:40.674376 21701 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:17:40.675323 21701 webserver.cc:460] Webserver started at http://127.0.0.1:36055/ using document root <none> and password file <none>
I20260812 03:17:40.675925 21701 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:40.676026 21701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:40.676470 21701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:40.678764 21701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "17c0fe6cc6914a3d92642e667dc05d71"
format_stamp: "Formatted at 2026-08-12 03:17:40 on dist-test-slave-7nm7"
server_key: "2b3660bab008f7d7eb4daf7b9dd48b71"
server_key_iv: "76f003501322caa05e43e1c678ac0755"
server_key_version: "encryptionkey@0"
I20260812 03:17:40.679483 21701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "17c0fe6cc6914a3d92642e667dc05d71"
format_stamp: "Formatted at 2026-08-12 03:17:40 on dist-test-slave-7nm7"
server_key: "2b3660bab008f7d7eb4daf7b9dd48b71"
server_key_iv: "76f003501322caa05e43e1c678ac0755"
server_key_version: "encryptionkey@0"
I20260812 03:17:40.683375 21701 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:17:40.686429 21722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:40.687703 21701 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:17:40.687853 21701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "17c0fe6cc6914a3d92642e667dc05d71"
format_stamp: "Formatted at 2026-08-12 03:17:40 on dist-test-slave-7nm7"
server_key: "2b3660bab008f7d7eb4daf7b9dd48b71"
server_key_iv: "76f003501322caa05e43e1c678ac0755"
server_key_version: "encryptionkey@0"
I20260812 03:17:40.687975 21701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:40.698207 21701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:40.701256 21701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:40.701469 21701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:40.709219 21701 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37527
I20260812 03:17:40.709249 21823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37527 every 8 connection(s)
I20260812 03:17:40.710202 21701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:17:40.713065 21824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:40.717676 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 21701
I20260812 03:17:40.717779 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:17:40.718101 17253 external_mini_cluster.cc:1428] Setting key 011c4a909a22ddfdc1678551b7fea15b
I20260812 03:17:40.718835 21824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71: Bootstrap starting.
I20260812 03:17:40.721318 21824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:40.722060 21824 log.cc:824] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:40.724393 21824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71: No bootstrap required, opened a new log
Aug 12 03:17:40 dist-test-slave-7nm7 krb5kdc[21675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:17:40.728736 21824 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17c0fe6cc6914a3d92642e667dc05d71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37527 } }
I20260812 03:17:40.729045 21824 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:40.729125 21824 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17c0fe6cc6914a3d92642e667dc05d71, State: Initialized, Role: FOLLOWER
I20260812 03:17:40.729635 21824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17c0fe6cc6914a3d92642e667dc05d71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37527 } }
I20260812 03:17:40.729764 21824 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:17:40.729847 21824 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:17:40.729946 21824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:40.730903 21824 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17c0fe6cc6914a3d92642e667dc05d71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37527 } }
I20260812 03:17:40.731266 21824 leader_election.cc:302] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17c0fe6cc6914a3d92642e667dc05d71; no voters:
I20260812 03:17:40.731621 21824 leader_election.cc:288] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:17:40.731693 21835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:40.731935 21835 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71 [term 1 LEADER]: Becoming Leader. State: Replica: 17c0fe6cc6914a3d92642e667dc05d71, State: Running, Role: LEADER
I20260812 03:17:40.732231 21835 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17c0fe6cc6914a3d92642e667dc05d71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37527 } }
I20260812 03:17:40.733006 21824 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:17:40.733433 21835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17c0fe6cc6914a3d92642e667dc05d71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17c0fe6cc6914a3d92642e667dc05d71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37527 } } }
I20260812 03:17:40.733543 21835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:40.733860 21838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17c0fe6cc6914a3d92642e667dc05d71. Latest consensus state: current_term: 1 leader_uuid: "17c0fe6cc6914a3d92642e667dc05d71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17c0fe6cc6914a3d92642e667dc05d71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37527 } } }
I20260812 03:17:40.733947 21838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:40.734462 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:40.719742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35415 (local address 127.0.0.1:37527)
0812 03:17:40.720195 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:40.720211 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:40.720246 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:17:40.720874 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:40.720890 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:40.721211 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:40.721537 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:40.722360 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:40.723754 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:40.724653 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:40.725328 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:40.725506 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:40.728327 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:40.728333 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:40.728350 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:40.728381 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:40.730512 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:40.731042 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:40.731048 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:40.731062 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:40.731183 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:40.731556 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:40.731559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:40.731561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:40.732104 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:40.732933 (+ 829us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:40.733105 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 03:17:40.733373 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":145,"threads_started":1}
I20260812 03:17:40.737715 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 03:17:40.738286 21857 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:17:40.738384 21857 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:17:40.738471 21841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:17:40.739259 21841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:17:40.744949 21841 catalog_manager.cc:1347] Generated new cluster ID: fc2b120463074fb18a631ddc232653f1
I20260812 03:17:40.745012 21841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:17:40.750051 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:40.740016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37443 (local address 127.0.0.1:37527)
0812 03:17:40.740124 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:40.740128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:40.741201 (+ 1073us) server_negotiation.cc:407] Connection header received
0812 03:17:40.741303 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:40.741306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:40.741355 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:40.741444 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:40.742306 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:40.743533 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:40.744621 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:40.744983 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:40.745090 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:40.746635 (+ 1545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:40.746639 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:40.746642 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:40.746667 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:40.748105 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:40.748728 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:40.748733 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:40.748734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:40.748829 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:40.749173 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:40.749176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:40.749178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:40.749355 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:40.749512 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:40.749788 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 03:17:40.749880 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:17:40.757480 21841 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:17:40.758462 21841 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:17:40.766407 21841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17c0fe6cc6914a3d92642e667dc05d71: Generated new TSK 0
I20260812 03:17:40.767203 21841 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:17:40.774451 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 21701
W20260812 03:17:40.788571 21673 connection.cc:535] client connection to 127.0.0.1:37527 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37527): Connection reset by peer (error 104)
2026-08-12T03:17:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1697 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:17:40 dist-test-slave-7nm7 krb5kdc[21880](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:17:40 dist-test-slave-7nm7 krb5kdc[21880](info): set up 2 sockets
Aug 12 03:17:40 dist-test-slave-7nm7 krb5kdc[21880](info): commencing operation
krb5kdc: starting...
W20260812 03:17:42.091073 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:17:42 dist-test-slave-7nm7 krb5kdc[21880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:17:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:17:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:17:42.228979 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37493
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:42.354111 21907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:42.354575 21907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:42.354624 21907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:42.359949 21907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:42.360034 21907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:42.360286 21907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:42.360373 21907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:42.360419 21907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:42.367122 21907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37493
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:42.368335 21907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:42.369488 21907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:42.378633 21915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:42.378665 21918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:42.378644 21907 server_base.cc:1034] running on GCE node
W20260812 03:17:42.378666 21921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:42.379510 21907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:42.380630 21907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:42.381817 21907 hybrid_clock.cc:648] HybridClock initialized: now 1786504662381740 us; error 91 us; skew 500 ppm
Aug 12 03:17:42 dist-test-slave-7nm7 krb5kdc[21880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504662, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:42.388386 21907 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:17:42.389382 21907 webserver.cc:460] Webserver started at http://127.0.0.1:45791/ using document root <none> and password file <none>
I20260812 03:17:42.389986 21907 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:42.390049 21907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:42.390327 21907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:42.392480 21907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "43848ba669074bfeb24488941f2d1256"
format_stamp: "Formatted at 2026-08-12 03:17:42 on dist-test-slave-7nm7"
server_key: "eb7bf5eb9224aaa57bdf8188f145d8db"
server_key_iv: "41b9c28d53f4322f3871bae4314d1ead"
server_key_version: "encryptionkey@0"
I20260812 03:17:42.393092 21907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "43848ba669074bfeb24488941f2d1256"
format_stamp: "Formatted at 2026-08-12 03:17:42 on dist-test-slave-7nm7"
server_key: "eb7bf5eb9224aaa57bdf8188f145d8db"
server_key_iv: "41b9c28d53f4322f3871bae4314d1ead"
server_key_version: "encryptionkey@0"
I20260812 03:17:42.397445 21907 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:17:42.400470 21933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:42.401785 21907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 03:17:42.401969 21907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "43848ba669074bfeb24488941f2d1256"
format_stamp: "Formatted at 2026-08-12 03:17:42 on dist-test-slave-7nm7"
server_key: "eb7bf5eb9224aaa57bdf8188f145d8db"
server_key_iv: "41b9c28d53f4322f3871bae4314d1ead"
server_key_version: "encryptionkey@0"
I20260812 03:17:42.402102 21907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:42.427666 21907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:42.431164 21907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:42.431404 21907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:42.439465 21907 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37493
I20260812 03:17:42.439504 22030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37493 every 8 connection(s)
I20260812 03:17:42.440912 21907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:17:42.445111 22033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:42.447196 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 21907
I20260812 03:17:42.447361 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:17:42.447651 17253 external_mini_cluster.cc:1428] Setting key c151dfc1b80e808f51f5aba2db6ff2f1
I20260812 03:17:42.451946 22033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256: Bootstrap starting.
I20260812 03:17:42.454413 22033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:42.455248 22033 log.cc:824] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:42.457001 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:42.449099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43007 (local address 127.0.0.1:37493)
0812 03:17:42.449567 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:42.449582 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:42.449618 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:17:42.450377 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:42.450396 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:42.450780 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:42.451109 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:42.451534 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:42.452525 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:42.453420 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:42.453896 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:42.454075 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:42.455086 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:42.455092 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:42.455105 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:42.455560 (+ 455us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:42.455617 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:42.455640 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:42.455789 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:42.455974 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 03:17:42.456121 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":134,"threads_started":1}
I20260812 03:17:42.457520 22033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256: No bootstrap required, opened a new log
I20260812 03:17:42.460266 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:17:42.461709 22033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43848ba669074bfeb24488941f2d1256" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37493 } }
I20260812 03:17:42.462016 22033 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:42.462080 22033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43848ba669074bfeb24488941f2d1256, State: Initialized, Role: FOLLOWER
I20260812 03:17:42.462584 22033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43848ba669074bfeb24488941f2d1256" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37493 } }
I20260812 03:17:42.462766 22033 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:17:42.462831 22033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:17:42.462946 22033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:42.464080 22033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43848ba669074bfeb24488941f2d1256" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37493 } }
I20260812 03:17:42.464473 22033 leader_election.cc:302] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43848ba669074bfeb24488941f2d1256; no voters:
I20260812 03:17:42.464870 22033 leader_election.cc:288] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:17:42.464993 22053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:42.465260 22053 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256 [term 1 LEADER]: Becoming Leader. State: Replica: 43848ba669074bfeb24488941f2d1256, State: Running, Role: LEADER
I20260812 03:17:42.465577 22053 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43848ba669074bfeb24488941f2d1256" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37493 } }
I20260812 03:17:42.466964 22054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43848ba669074bfeb24488941f2d1256. Latest consensus state: current_term: 1 leader_uuid: "43848ba669074bfeb24488941f2d1256" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43848ba669074bfeb24488941f2d1256" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37493 } } }
I20260812 03:17:42.467115 22054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:42.467262 22033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:17:42.467478 22055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43848ba669074bfeb24488941f2d1256" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43848ba669074bfeb24488941f2d1256" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37493 } } }
I20260812 03:17:42.467574 22055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:42.468084 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:42.462657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45959 (local address 127.0.0.1:37493)
0812 03:17:42.462781 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:42.462786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:42.462926 (+ 140us) server_negotiation.cc:407] Connection header received
0812 03:17:42.463121 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:42.463123 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:42.463162 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:42.463238 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:42.463570 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:42.464112 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:42.465858 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:42.466202 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:42.466341 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:42.467523 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:42.467527 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:42.467529 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:42.467544 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:42.467573 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:42.467588 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:42.467680 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:42.467901 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 03:17:42.467944 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:17:42.468403 22057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:17:42.471978 22057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:17:42.477429 22057 catalog_manager.cc:1347] Generated new cluster ID: 022a0c386b53412291c43f67d3b19a2e
I20260812 03:17:42.477510 22057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:17:42.496625 22057 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:17:42.497927 22057 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:17:42.513017 22057 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43848ba669074bfeb24488941f2d1256: Generated new TSK 0
I20260812 03:17:42.513816 22057 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:17:42.519457 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 21907
W20260812 03:17:42.536288 21877 connection.cc:535] client connection to 127.0.0.1:37493 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37493): Connection reset by peer (error 104)
2026-08-12T03:17:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1747 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:17:42 dist-test-slave-7nm7 krb5kdc[22089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:17:42 dist-test-slave-7nm7 krb5kdc[22089](info): set up 2 sockets
Aug 12 03:17:42 dist-test-slave-7nm7 krb5kdc[22089](info): commencing operation
krb5kdc: starting...
W20260812 03:17:43.833716 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:17:43 dist-test-slave-7nm7 krb5kdc[22089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:17:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:17:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:17:43.974830 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45065
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:44.099452 22115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:44.099753 22115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:44.099797 22115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:44.105383 22115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:44.105458 22115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:44.105479 22115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:44.105495 22115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:44.105511 22115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:44.111037 22115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45065
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:44.112134 22115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:44.113133 22115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:44.122592 22121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:44.122717 22125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:44.122612 22122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:44.123299 22115 server_base.cc:1034] running on GCE node
I20260812 03:17:44.123978 22115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:44.125164 22115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:44.126348 22115 hybrid_clock.cc:648] HybridClock initialized: now 1786504664126320 us; error 41 us; skew 500 ppm
Aug 12 03:17:44 dist-test-slave-7nm7 krb5kdc[22089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504664, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:44.129282 22115 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:17:44.130155 22115 webserver.cc:460] Webserver started at http://127.0.0.1:35271/ using document root <none> and password file <none>
I20260812 03:17:44.130751 22115 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:44.130813 22115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:44.130980 22115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:44.133291 22115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "7c3feb2405314df78c0bd2f303b762ec"
format_stamp: "Formatted at 2026-08-12 03:17:44 on dist-test-slave-7nm7"
server_key: "02e13e66e38c8589ec2059538170f820"
server_key_iv: "cf792de77c180bdad9e2fc022ef4d8b8"
server_key_version: "encryptionkey@0"
I20260812 03:17:44.133836 22115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "7c3feb2405314df78c0bd2f303b762ec"
format_stamp: "Formatted at 2026-08-12 03:17:44 on dist-test-slave-7nm7"
server_key: "02e13e66e38c8589ec2059538170f820"
server_key_iv: "cf792de77c180bdad9e2fc022ef4d8b8"
server_key_version: "encryptionkey@0"
I20260812 03:17:44.138212 22115 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 03:17:44.141075 22145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:44.142611 22115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 03:17:44.142776 22115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "7c3feb2405314df78c0bd2f303b762ec"
format_stamp: "Formatted at 2026-08-12 03:17:44 on dist-test-slave-7nm7"
server_key: "02e13e66e38c8589ec2059538170f820"
server_key_iv: "cf792de77c180bdad9e2fc022ef4d8b8"
server_key_version: "encryptionkey@0"
I20260812 03:17:44.142884 22115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:44.147742 22115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:44.151276 22115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:44.151434 22115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:44.159134 22115 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45065
I20260812 03:17:44.159164 22228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45065 every 8 connection(s)
I20260812 03:17:44.160617 22115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:17:44.163410 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 22115
I20260812 03:17:44.163578 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:17:44.163861 22231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:44.163889 17253 external_mini_cluster.cc:1428] Setting key 28cb144cc9a6afa3c60a7379ab5ad20a
I20260812 03:17:44.169410 22235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:44.165308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49021 (local address 127.0.0.1:45065)
0812 03:17:44.165804 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:44.165820 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:44.165855 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:17:44.166705 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:44.166724 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:44.167009 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:44.167364 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:44.168218 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:44.168222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:44.168234 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:44.168246 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:44.168301 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:44.168322 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:44.168482 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:44.168552 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 03:17:44.168714 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":167,"threads_started":1}
I20260812 03:17:44.171299 22231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec: Bootstrap starting.
I20260812 03:17:44.172425 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:17:44.173983 22231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:44.175144 22231 log.cc:824] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:44.176839 22235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:44.174752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34179 (local address 127.0.0.1:45065)
0812 03:17:44.174897 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:44.174902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:44.175183 (+ 281us) server_negotiation.cc:407] Connection header received
0812 03:17:44.175306 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:44.175310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:44.175350 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:44.175434 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:44.176363 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:44.176366 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:44.176368 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:44.176379 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:44.176405 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:44.176415 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:44.176495 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:44.176621 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 03:17:44.176655 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:17:44.177893 22231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec: No bootstrap required, opened a new log
I20260812 03:17:44.181625 22231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3feb2405314df78c0bd2f303b762ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45065 } }
I20260812 03:17:44.181895 22231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:44.181973 22231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3feb2405314df78c0bd2f303b762ec, State: Initialized, Role: FOLLOWER
I20260812 03:17:44.182494 22231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3feb2405314df78c0bd2f303b762ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45065 } }
I20260812 03:17:44.182687 22231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:17:44.182766 22231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:17:44.182873 22231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:44.184072 22231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3feb2405314df78c0bd2f303b762ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45065 } }
I20260812 03:17:44.184433 22231 leader_election.cc:302] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c3feb2405314df78c0bd2f303b762ec; no voters:
I20260812 03:17:44.184796 22231 leader_election.cc:288] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:17:44.185043 22252 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:44.185310 22252 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec [term 1 LEADER]: Becoming Leader. State: Replica: 7c3feb2405314df78c0bd2f303b762ec, State: Running, Role: LEADER
I20260812 03:17:44.185622 22252 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3feb2405314df78c0bd2f303b762ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45065 } }
I20260812 03:17:44.186323 22231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:17:44.187404 22253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c3feb2405314df78c0bd2f303b762ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3feb2405314df78c0bd2f303b762ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45065 } } }
I20260812 03:17:44.187546 22253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:44.187827 22254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c3feb2405314df78c0bd2f303b762ec. Latest consensus state: current_term: 1 leader_uuid: "7c3feb2405314df78c0bd2f303b762ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3feb2405314df78c0bd2f303b762ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45065 } } }
I20260812 03:17:44.187922 22254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:44.187994 22271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:17:44.191335 22271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:17:44.197811 22271 catalog_manager.cc:1347] Generated new cluster ID: 5f913562bb844e1898b0e892f17dddb1
I20260812 03:17:44.198016 22271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:17:44.215816 22271 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:17:44.216257 22271 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:17:44.225497 22271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c3feb2405314df78c0bd2f303b762ec: Generated new TSK 0
I20260812 03:17:44.226363 22271 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:17:44.231765 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 22115
2026-08-12T03:17:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1708 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:17:44 dist-test-slave-7nm7 krb5kdc[22289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:17:44 dist-test-slave-7nm7 krb5kdc[22289](info): set up 2 sockets
Aug 12 03:17:44 dist-test-slave-7nm7 krb5kdc[22289](info): commencing operation
krb5kdc: starting...
W20260812 03:17:45.562170 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:17:45 dist-test-slave-7nm7 krb5kdc[22289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:17:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:17:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:17:45.711642 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46523
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:45.836979 22318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:45.837260 22318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:45.837302 22318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:45.841439 22318 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:45.841511 22318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:45.841531 22318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:45.841552 22318 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:45.841612 22318 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:45.847076 22318 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:45.848227 22318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:45.849376 22318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:45.858122 22333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:45.858073 22330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:45.858165 22327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:45.858222 22318 server_base.cc:1034] running on GCE node
I20260812 03:17:45.859045 22318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:45.860296 22318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:45.861509 22318 hybrid_clock.cc:648] HybridClock initialized: now 1786504665861478 us; error 44 us; skew 500 ppm
Aug 12 03:17:45 dist-test-slave-7nm7 krb5kdc[22289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504665, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:45.864530 22318 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:17:45.865500 22318 webserver.cc:460] Webserver started at http://127.0.0.1:40481/ using document root <none> and password file <none>
I20260812 03:17:45.866096 22318 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:45.866194 22318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:45.866472 22318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:45.868613 22318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "7805d62caf7f468dadf6091a5287e70a"
format_stamp: "Formatted at 2026-08-12 03:17:45 on dist-test-slave-7nm7"
server_key: "3d16abdc7fb0b60d784d8ccdbbdd42f5"
server_key_iv: "f172cc0dc13447be520ea06f58a67e8d"
server_key_version: "encryptionkey@0"
I20260812 03:17:45.869220 22318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "7805d62caf7f468dadf6091a5287e70a"
format_stamp: "Formatted at 2026-08-12 03:17:45 on dist-test-slave-7nm7"
server_key: "3d16abdc7fb0b60d784d8ccdbbdd42f5"
server_key_iv: "f172cc0dc13447be520ea06f58a67e8d"
server_key_version: "encryptionkey@0"
I20260812 03:17:45.873178 22318 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260812 03:17:45.875790 22348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:45.877107 22318 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:17:45.877292 22318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "7805d62caf7f468dadf6091a5287e70a"
format_stamp: "Formatted at 2026-08-12 03:17:45 on dist-test-slave-7nm7"
server_key: "3d16abdc7fb0b60d784d8ccdbbdd42f5"
server_key_iv: "f172cc0dc13447be520ea06f58a67e8d"
server_key_version: "encryptionkey@0"
I20260812 03:17:45.877420 22318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:45.899946 22318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:45.903723 22318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:45.903949 22318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:45.911919 22318 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46523
I20260812 03:17:45.912017 22437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46523 every 8 connection(s)
I20260812 03:17:45.912935 22318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:17:45.915915 22439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:45.920259 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 22318
I20260812 03:17:45.920410 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:17:45.920717 17253 external_mini_cluster.cc:1428] Setting key 173c81f6559a9c275267a6e791f768df
I20260812 03:17:45.922297 22439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a: Bootstrap starting.
I20260812 03:17:45.924904 22439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:45.925727 22439 log.cc:824] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:45.927934 22439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a: No bootstrap required, opened a new log
Aug 12 03:17:45 dist-test-slave-7nm7 krb5kdc[22289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:17:45.932411 22439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7805d62caf7f468dadf6091a5287e70a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46523 } }
I20260812 03:17:45.932672 22439 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:45.932818 22439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7805d62caf7f468dadf6091a5287e70a, State: Initialized, Role: FOLLOWER
I20260812 03:17:45.933908 22439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7805d62caf7f468dadf6091a5287e70a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46523 } }
I20260812 03:17:45.934119 22439 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:17:45.934222 22439 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:17:45.934376 22439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:45.935513 22439 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7805d62caf7f468dadf6091a5287e70a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46523 } }
I20260812 03:17:45.935938 22439 leader_election.cc:302] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7805d62caf7f468dadf6091a5287e70a; no voters:
I20260812 03:17:45.936326 22439 leader_election.cc:288] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:17:45.936450 22450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:45.936667 22450 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a [term 1 LEADER]: Becoming Leader. State: Replica: 7805d62caf7f468dadf6091a5287e70a, State: Running, Role: LEADER
I20260812 03:17:45.937050 22450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7805d62caf7f468dadf6091a5287e70a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46523 } }
I20260812 03:17:45.937515 22447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:45.922421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59679 (local address 127.0.0.1:46523)
0812 03:17:45.922891 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:45.922912 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:45.922947 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:17:45.923626 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:45.923643 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:45.923982 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:45.924324 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:45.925235 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:45.926360 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:45.927232 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:45.927711 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:45.927964 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:45.930739 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:45.930745 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:45.930759 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:45.930787 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:45.932610 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:45.933531 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:45.933539 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:45.933547 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:45.933706 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:45.934221 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:45.934230 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:45.934232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:45.935088 (+ 856us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:45.935457 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:45.936076 (+ 619us) server_negotiation.cc:299] Negotiation successful
0812 03:17:45.936450 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":142,"threads_started":1}
I20260812 03:17:45.937645 22439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:17:45.938880 22454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7805d62caf7f468dadf6091a5287e70a. Latest consensus state: current_term: 1 leader_uuid: "7805d62caf7f468dadf6091a5287e70a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7805d62caf7f468dadf6091a5287e70a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46523 } } }
I20260812 03:17:45.939016 22454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:45.939357 22453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7805d62caf7f468dadf6091a5287e70a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7805d62caf7f468dadf6091a5287e70a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46523 } } }
I20260812 03:17:45.939445 22453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:45.939719 22468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:17:45.941715 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:17:45.943713 22468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:17:45.951803 22468 catalog_manager.cc:1347] Generated new cluster ID: 88d996a4efbc4a42833b446e6aa9bba1
I20260812 03:17:45.951890 22468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:17:45.953109 22447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:45.944221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49031 (local address 127.0.0.1:46523)
0812 03:17:45.944344 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:45.944350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:45.944645 (+ 295us) server_negotiation.cc:407] Connection header received
0812 03:17:45.944871 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:45.944875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:45.944927 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:45.945018 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:45.945891 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:45.946542 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:45.947383 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:45.947636 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:45.947750 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:45.949400 (+ 1650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:45.949404 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:45.949406 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:45.949428 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:45.951266 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:45.951792 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:45.951795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:45.951797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:45.951920 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:45.952218 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:45.952223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:45.952225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:45.952416 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:45.952568 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:45.952816 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 03:17:45.952943 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:17:45.964074 22468 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:17:45.965423 22468 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:17:45.983065 22468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7805d62caf7f468dadf6091a5287e70a: Generated new TSK 0
I20260812 03:17:45.984356 22468 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:17:45.991511 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 22318
2026-08-12T03:17:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1773 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:17:46 dist-test-slave-7nm7 krb5kdc[22499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:17:46 dist-test-slave-7nm7 krb5kdc[22499](info): set up 2 sockets
Aug 12 03:17:46 dist-test-slave-7nm7 krb5kdc[22499](info): commencing operation
krb5kdc: starting...
W20260812 03:17:47.349876 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:17:47 dist-test-slave-7nm7 krb5kdc[22499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:17:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:17:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:17:47.493325 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44249
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:47.620847 22523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:47.621130 22523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:47.621178 22523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:47.625481 22523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:47.625556 22523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:47.625576 22523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:47.625594 22523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:47.625622 22523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:47.631563 22523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:47.632772 22523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:47.633886 22523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:47.643875 22542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:47.643873 22540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:47.643906 22545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:47.644452 22523 server_base.cc:1034] running on GCE node
I20260812 03:17:47.645143 22523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:47.646418 22523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:47.647606 22523 hybrid_clock.cc:648] HybridClock initialized: now 1786504667647585 us; error 34 us; skew 500 ppm
Aug 12 03:17:47 dist-test-slave-7nm7 krb5kdc[22499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504667, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:47.653925 22523 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:17:47.655440 22523 webserver.cc:460] Webserver started at http://127.0.0.1:41789/ using document root <none> and password file <none>
I20260812 03:17:47.656503 22523 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:47.656599 22523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:47.656869 22523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:47.659155 22523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "638abe526a984c0884a3b4170a0fb386"
format_stamp: "Formatted at 2026-08-12 03:17:47 on dist-test-slave-7nm7"
server_key: "195c6755fe9e60cb9f54074824b0bdc3"
server_key_iv: "a945b58141b4b29b8007dd523d0ac9bf"
server_key_version: "encryptionkey@0"
I20260812 03:17:47.659740 22523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "638abe526a984c0884a3b4170a0fb386"
format_stamp: "Formatted at 2026-08-12 03:17:47 on dist-test-slave-7nm7"
server_key: "195c6755fe9e60cb9f54074824b0bdc3"
server_key_iv: "a945b58141b4b29b8007dd523d0ac9bf"
server_key_version: "encryptionkey@0"
I20260812 03:17:47.664177 22523 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:17:47.666836 22557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:47.668125 22523 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:17:47.668265 22523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "638abe526a984c0884a3b4170a0fb386"
format_stamp: "Formatted at 2026-08-12 03:17:47 on dist-test-slave-7nm7"
server_key: "195c6755fe9e60cb9f54074824b0bdc3"
server_key_iv: "a945b58141b4b29b8007dd523d0ac9bf"
server_key_version: "encryptionkey@0"
I20260812 03:17:47.668417 22523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:47.683275 22523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:47.686894 22523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:47.687139 22523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:47.695604 22523 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44249
I20260812 03:17:47.695631 22635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44249 every 8 connection(s)
I20260812 03:17:47.697009 22523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:17:47.700071 22641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:47.701750 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 22523
I20260812 03:17:47.701929 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:17:47.702378 17253 external_mini_cluster.cc:1428] Setting key 33764d7fd4b44ae1b57e2d620e9a97e9
I20260812 03:17:47.707226 22641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386: Bootstrap starting.
I20260812 03:17:47.710124 22641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:47.711072 22641 log.cc:824] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:47.712646 22646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:47.704224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49317 (local address 127.0.0.1:44249)
0812 03:17:47.704697 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:47.704728 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:47.704766 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:17:47.705415 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:47.705432 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:47.705773 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:47.706057 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:47.706498 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:47.707701 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:47.709141 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:47.709643 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:47.709771 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:47.710742 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:47.710747 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:47.710761 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:47.711249 (+ 488us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:47.711310 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:47.711336 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:47.711423 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:47.711622 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 03:17:47.711847 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":154,"threads_started":1}
I20260812 03:17:47.713264 22641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386: No bootstrap required, opened a new log
I20260812 03:17:47.715871 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:17:47.718101 22641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "638abe526a984c0884a3b4170a0fb386" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44249 } }
I20260812 03:17:47.718446 22641 raft_consensus.cc:374] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:47.718515 22641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 638abe526a984c0884a3b4170a0fb386, State: Initialized, Role: FOLLOWER
I20260812 03:17:47.719035 22641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "638abe526a984c0884a3b4170a0fb386" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44249 } }
I20260812 03:17:47.719210 22641 raft_consensus.cc:388] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:17:47.719300 22641 raft_consensus.cc:482] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:17:47.719413 22641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:47.720818 22641 raft_consensus.cc:504] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "638abe526a984c0884a3b4170a0fb386" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44249 } }
I20260812 03:17:47.721217 22641 leader_election.cc:302] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 638abe526a984c0884a3b4170a0fb386; no voters:
I20260812 03:17:47.721675 22641 leader_election.cc:288] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:17:47.721752 22654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:47.721982 22654 raft_consensus.cc:686] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386 [term 1 LEADER]: Becoming Leader. State: Replica: 638abe526a984c0884a3b4170a0fb386, State: Running, Role: LEADER
I20260812 03:17:47.722329 22654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "638abe526a984c0884a3b4170a0fb386" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44249 } }
I20260812 03:17:47.723161 22641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:17:47.723402 22646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:47.718282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57007 (local address 127.0.0.1:44249)
0812 03:17:47.718402 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:47.718408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:47.718661 (+ 253us) server_negotiation.cc:407] Connection header received
0812 03:17:47.718848 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:47.718852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:47.718906 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:47.718974 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:47.719318 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:47.720053 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:47.721394 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:47.721684 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:47.721827 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:47.722943 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:47.722946 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:47.722948 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:47.722960 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:47.722991 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:47.723005 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:47.723096 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:47.723204 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 03:17:47.723260 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:17:47.724566 22657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "638abe526a984c0884a3b4170a0fb386" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "638abe526a984c0884a3b4170a0fb386" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44249 } } }
I20260812 03:17:47.724943 22657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:47.724043 22655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 638abe526a984c0884a3b4170a0fb386. Latest consensus state: current_term: 1 leader_uuid: "638abe526a984c0884a3b4170a0fb386" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "638abe526a984c0884a3b4170a0fb386" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44249 } } }
I20260812 03:17:47.725333 22655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:47.725438 22674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:17:47.729367 22674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:17:47.735476 22674 catalog_manager.cc:1347] Generated new cluster ID: ff015ca4091448d5bad86de77f994aca
I20260812 03:17:47.735570 22674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:17:47.750171 22674 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:17:47.751276 22674 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:17:47.760169 22674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 638abe526a984c0884a3b4170a0fb386: Generated new TSK 0
I20260812 03:17:47.760949 22674 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:17:47.766535 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 22523
2026-08-12T03:17:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1768 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:17:47 dist-test-slave-7nm7 krb5kdc[22700](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:17:47 dist-test-slave-7nm7 krb5kdc[22700](info): set up 2 sockets
Aug 12 03:17:47 dist-test-slave-7nm7 krb5kdc[22700](info): commencing operation
krb5kdc: starting...
W20260812 03:17:49.161782 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.360s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:17:49 dist-test-slave-7nm7 krb5kdc[22700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:17:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:17:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:17:49.309980 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44427
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:49.429803 22718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:49.430135 22718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:49.430282 22718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:49.434422 22718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:49.434509 22718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:49.434546 22718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:49.434603 22718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:49.434641 22718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:49.440275 22718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:49.441478 22718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:49.442634 22718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:49.451517 22728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:49.451552 22727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:49.451542 22731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:49.451767 22718 server_base.cc:1034] running on GCE node
I20260812 03:17:49.452340 22718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:49.453457 22718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:49.454655 22718 hybrid_clock.cc:648] HybridClock initialized: now 1786504669454641 us; error 26 us; skew 500 ppm
Aug 12 03:17:49 dist-test-slave-7nm7 krb5kdc[22700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504669, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:49.457810 22718 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:17:49.458868 22718 webserver.cc:460] Webserver started at http://127.0.0.1:42487/ using document root <none> and password file <none>
I20260812 03:17:49.459407 22718 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:49.459468 22718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:49.459645 22718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:49.461689 22718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "3ce3f7a791114832ba8c8d1c3de18f58"
format_stamp: "Formatted at 2026-08-12 03:17:49 on dist-test-slave-7nm7"
server_key: "60519cc20bba16b11122f984c07d97d7"
server_key_iv: "ac3faea1220a2d0a55388893b2084495"
server_key_version: "encryptionkey@0"
I20260812 03:17:49.462210 22718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "3ce3f7a791114832ba8c8d1c3de18f58"
format_stamp: "Formatted at 2026-08-12 03:17:49 on dist-test-slave-7nm7"
server_key: "60519cc20bba16b11122f984c07d97d7"
server_key_iv: "ac3faea1220a2d0a55388893b2084495"
server_key_version: "encryptionkey@0"
I20260812 03:17:49.466464 22718 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:17:49.469261 22744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:49.470750 22718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:17:49.470896 22718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "3ce3f7a791114832ba8c8d1c3de18f58"
format_stamp: "Formatted at 2026-08-12 03:17:49 on dist-test-slave-7nm7"
server_key: "60519cc20bba16b11122f984c07d97d7"
server_key_iv: "ac3faea1220a2d0a55388893b2084495"
server_key_version: "encryptionkey@0"
I20260812 03:17:49.471009 22718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:49.475797 22718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:49.479409 22718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:49.479630 22718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:49.487850 22718 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44427
I20260812 03:17:49.487936 22833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44427 every 8 connection(s)
I20260812 03:17:49.488910 22718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:17:49.492117 22835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:49.497975 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 22718
I20260812 03:17:49.498083 22835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58: Bootstrap starting.
I20260812 03:17:49.498139 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:17:49.498510 17253 external_mini_cluster.cc:1428] Setting key 4a7bb6e821903c9b3b08d3aeea57bdfd
I20260812 03:17:49.500830 22835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:49.501603 22835 log.cc:824] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:49.503870 22840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:49.500012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34905 (local address 127.0.0.1:44427)
0812 03:17:49.500479 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:49.500494 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:49.500528 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:17:49.501213 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:49.501231 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:49.501505 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:49.501864 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:49.502701 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:49.502711 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:49.502723 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:49.502734 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:49.502787 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:49.502807 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:49.502929 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:49.503030 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 03:17:49.503199 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":137,"threads_started":1}
I20260812 03:17:49.504053 22835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58: No bootstrap required, opened a new log
I20260812 03:17:49.506916 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:17:49.508591 22835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce3f7a791114832ba8c8d1c3de18f58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44427 } }
I20260812 03:17:49.508853 22835 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:49.508919 22835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce3f7a791114832ba8c8d1c3de18f58, State: Initialized, Role: FOLLOWER
I20260812 03:17:49.509446 22835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce3f7a791114832ba8c8d1c3de18f58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44427 } }
I20260812 03:17:49.509647 22835 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:17:49.509724 22835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:17:49.509846 22835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:49.511045 22840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:49.508977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57133 (local address 127.0.0.1:44427)
0812 03:17:49.509108 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:49.509113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:49.509348 (+ 235us) server_negotiation.cc:407] Connection header received
0812 03:17:49.509534 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:49.509537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:49.509576 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:49.509794 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:49.510618 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:49.510621 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:49.510623 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:49.510634 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:49.510658 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:49.510668 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:49.510740 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:49.510869 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 03:17:49.510903 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:17:49.511409 22835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce3f7a791114832ba8c8d1c3de18f58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44427 } }
I20260812 03:17:49.511791 22835 leader_election.cc:302] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ce3f7a791114832ba8c8d1c3de18f58; no voters:
I20260812 03:17:49.512197 22835 leader_election.cc:288] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:17:49.512286 22852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:49.512540 22852 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce3f7a791114832ba8c8d1c3de18f58, State: Running, Role: LEADER
I20260812 03:17:49.512859 22852 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce3f7a791114832ba8c8d1c3de18f58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44427 } }
I20260812 03:17:49.513682 22835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:17:49.514616 22853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ce3f7a791114832ba8c8d1c3de18f58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce3f7a791114832ba8c8d1c3de18f58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44427 } } }
I20260812 03:17:49.514760 22853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:49.515091 22862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ce3f7a791114832ba8c8d1c3de18f58. Latest consensus state: current_term: 1 leader_uuid: "3ce3f7a791114832ba8c8d1c3de18f58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce3f7a791114832ba8c8d1c3de18f58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44427 } } }
I20260812 03:17:49.515192 22862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:49.515868 22870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:17:49.519168 22870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:17:49.525502 22870 catalog_manager.cc:1347] Generated new cluster ID: ebd06d3dd8b946798f24397c841451e8
I20260812 03:17:49.525619 22870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:17:49.539906 22870 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:17:49.540297 22870 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:17:49.546454 22870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ce3f7a791114832ba8c8d1c3de18f58: Generated new TSK 0
I20260812 03:17:49.547360 22870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:17:49.556197 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 22718
2026-08-12T03:17:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1785 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:17:49 dist-test-slave-7nm7 krb5kdc[22892](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:17:49 dist-test-slave-7nm7 krb5kdc[22892](info): set up 2 sockets
Aug 12 03:17:49 dist-test-slave-7nm7 krb5kdc[22892](info): commencing operation
krb5kdc: starting...
W20260812 03:17:50.890142 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:17:50 dist-test-slave-7nm7 krb5kdc[22892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:17:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:17:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:17:51.027292 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45461
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:51.144170 22918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:51.144484 22918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:51.144562 22918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:51.148677 22918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:51.148772 22918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:51.148810 22918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:51.148847 22918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:51.148880 22918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:51.154455 22918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:51.155607 22918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:51.156643 22918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:51.165498 22931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:51.165517 22933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:51.165601 22918 server_base.cc:1034] running on GCE node
W20260812 03:17:51.165611 22930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:51.166427 22918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:51.167600 22918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:51.168772 22918 hybrid_clock.cc:648] HybridClock initialized: now 1786504671168754 us; error 31 us; skew 500 ppm
Aug 12 03:17:51 dist-test-slave-7nm7 krb5kdc[22892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504671, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:51.172129 22918 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:17:51.173162 22918 webserver.cc:460] Webserver started at http://127.0.0.1:40139/ using document root <none> and password file <none>
I20260812 03:17:51.173764 22918 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:51.173874 22918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:51.174209 22918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:51.176427 22918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "2b4c9c9a45c5462384d63302a5aa772f"
format_stamp: "Formatted at 2026-08-12 03:17:51 on dist-test-slave-7nm7"
server_key: "8b72df551c2ac0c6b17a42c2684f5445"
server_key_iv: "7268a416c68a6a92c3851b0290f2a298"
server_key_version: "encryptionkey@0"
I20260812 03:17:51.177023 22918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "2b4c9c9a45c5462384d63302a5aa772f"
format_stamp: "Formatted at 2026-08-12 03:17:51 on dist-test-slave-7nm7"
server_key: "8b72df551c2ac0c6b17a42c2684f5445"
server_key_iv: "7268a416c68a6a92c3851b0290f2a298"
server_key_version: "encryptionkey@0"
I20260812 03:17:51.181537 22918 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 03:17:51.184316 22945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:51.186800 22918 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260812 03:17:51.186986 22918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "2b4c9c9a45c5462384d63302a5aa772f"
format_stamp: "Formatted at 2026-08-12 03:17:51 on dist-test-slave-7nm7"
server_key: "8b72df551c2ac0c6b17a42c2684f5445"
server_key_iv: "7268a416c68a6a92c3851b0290f2a298"
server_key_version: "encryptionkey@0"
I20260812 03:17:51.187150 22918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:51.214869 22918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:51.218199 22918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:51.218482 22918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:51.226605 22918 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45461
I20260812 03:17:51.226632 23039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45461 every 8 connection(s)
I20260812 03:17:51.227697 22918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:17:51.230974 23040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:51.235280 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 22918
I20260812 03:17:51.235435 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:17:51.235735 17253 external_mini_cluster.cc:1428] Setting key a158f57f3600eaec9b5068e842657e6f
I20260812 03:17:51.237435 23040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f: Bootstrap starting.
I20260812 03:17:51.239827 23040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:51.241366 23040 log.cc:824] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:51.243829 23040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f: No bootstrap required, opened a new log
Aug 12 03:17:51 dist-test-slave-7nm7 krb5kdc[22892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:17:51.248206 23040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4c9c9a45c5462384d63302a5aa772f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45461 } }
I20260812 03:17:51.248484 23040 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:51.248581 23040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b4c9c9a45c5462384d63302a5aa772f, State: Initialized, Role: FOLLOWER
I20260812 03:17:51.249089 23040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4c9c9a45c5462384d63302a5aa772f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45461 } }
I20260812 03:17:51.249253 23040 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:17:51.249353 23040 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:17:51.249465 23040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:51.250804 23040 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4c9c9a45c5462384d63302a5aa772f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45461 } }
I20260812 03:17:51.251255 23040 leader_election.cc:302] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b4c9c9a45c5462384d63302a5aa772f; no voters:
I20260812 03:17:51.251663 23040 leader_election.cc:288] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:17:51.252156 23052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:51.252473 23052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f [term 1 LEADER]: Becoming Leader. State: Replica: 2b4c9c9a45c5462384d63302a5aa772f, State: Running, Role: LEADER
I20260812 03:17:51.252532 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:51.237227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:51359 (local address 127.0.0.1:45461)
0812 03:17:51.237749 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:51.237766 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:51.237802 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:17:51.238562 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:51.238584 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:51.238931 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:51.239284 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:51.240209 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:51.241213 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:51.242400 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:51.242880 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:51.243070 (+ 190us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:51.245995 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:51.246000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:51.246015 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:51.246044 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:51.248836 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:51.249410 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:51.249416 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:51.249422 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:51.249576 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:51.249962 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:51.249966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:51.249967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:51.250606 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:51.250854 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:51.251190 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 03:17:51.251446 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":136,"threads_started":1}
I20260812 03:17:51.252839 23052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4c9c9a45c5462384d63302a5aa772f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45461 } }
I20260812 03:17:51.253091 23040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:17:51.255036 23053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b4c9c9a45c5462384d63302a5aa772f. Latest consensus state: current_term: 1 leader_uuid: "2b4c9c9a45c5462384d63302a5aa772f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4c9c9a45c5462384d63302a5aa772f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45461 } } }
I20260812 03:17:51.255194 23053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:51.255587 23049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b4c9c9a45c5462384d63302a5aa772f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4c9c9a45c5462384d63302a5aa772f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45461 } } }
I20260812 03:17:51.255685 23049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:51.258788 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 03:17:51.260429 23081 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:17:51.260610 23081 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:17:51.260787 23072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:17:51.262565 23072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:17:51.270002 23072 catalog_manager.cc:1347] Generated new cluster ID: 04d72a2552ed497cb9b80f517dc94200
I20260812 03:17:51.270092 23072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:17:51.274765 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:51.262442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:56951 (local address 127.0.0.1:45461)
0812 03:17:51.262563 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:51.262568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:51.265155 (+ 2587us) server_negotiation.cc:407] Connection header received
0812 03:17:51.265350 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:51.265354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:51.265415 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:51.265520 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:51.266355 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:51.266911 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:51.268032 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:51.268421 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:51.268538 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:51.270488 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:51.270493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:51.270496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:51.270525 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:51.272459 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:51.273303 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:51.273307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:51.273308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:51.273395 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:51.273778 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:51.273786 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:51.273788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:51.273972 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:51.274138 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:51.274473 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 03:17:51.274581 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:17:51.291818 23072 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:17:51.292925 23072 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:17:51.302647 23072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b4c9c9a45c5462384d63302a5aa772f: Generated new TSK 0
I20260812 03:17:51.303615 23072 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:17:51.310899 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 22918
W20260812 03:17:51.327829 22890 connection.cc:535] client connection to 127.0.0.1:45461 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45461): Connection reset by peer (error 104)
2026-08-12T03:17:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1754 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:17:51 dist-test-slave-7nm7 krb5kdc[23104](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:17:51 dist-test-slave-7nm7 krb5kdc[23104](info): set up 2 sockets
Aug 12 03:17:51 dist-test-slave-7nm7 krb5kdc[23104](info): commencing operation
krb5kdc: starting...
W20260812 03:17:52.665129 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.322s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:17:52 dist-test-slave-7nm7 krb5kdc[23104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:17:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:17:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:17:52.807420 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35935
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:52.926048 23131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:52.926364 23131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:52.926409 23131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:52.930474 23131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:52.930540 23131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:52.930560 23131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:52.930577 23131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:52.930620 23131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:52.936196 23131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:52.937319 23131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:52.938390 23131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:52.947642 23149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:52.947682 23143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:52.947835 23131 server_base.cc:1034] running on GCE node
W20260812 03:17:52.947841 23141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:52.948789 23131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:52.949976 23131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:52.951172 23131 hybrid_clock.cc:648] HybridClock initialized: now 1786504672951156 us; error 36 us; skew 500 ppm
Aug 12 03:17:52 dist-test-slave-7nm7 krb5kdc[23104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504672, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:52.959643 23131 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:17:52.960697 23131 webserver.cc:460] Webserver started at http://127.0.0.1:39121/ using document root <none> and password file <none>
I20260812 03:17:52.961303 23131 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:52.961369 23131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:52.961601 23131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:52.963804 23131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "3d21f19809304b44ac367f8f2ef6df2b"
format_stamp: "Formatted at 2026-08-12 03:17:52 on dist-test-slave-7nm7"
server_key: "918915d579b2497292226061dbb7832c"
server_key_iv: "488c26fefe0859b4fd8e8cd6c2d4ff63"
server_key_version: "encryptionkey@0"
I20260812 03:17:52.964399 23131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "3d21f19809304b44ac367f8f2ef6df2b"
format_stamp: "Formatted at 2026-08-12 03:17:52 on dist-test-slave-7nm7"
server_key: "918915d579b2497292226061dbb7832c"
server_key_iv: "488c26fefe0859b4fd8e8cd6c2d4ff63"
server_key_version: "encryptionkey@0"
I20260812 03:17:52.968322 23131 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:17:52.970894 23163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:52.972160 23131 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:17:52.972309 23131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "3d21f19809304b44ac367f8f2ef6df2b"
format_stamp: "Formatted at 2026-08-12 03:17:52 on dist-test-slave-7nm7"
server_key: "918915d579b2497292226061dbb7832c"
server_key_iv: "488c26fefe0859b4fd8e8cd6c2d4ff63"
server_key_version: "encryptionkey@0"
I20260812 03:17:52.972426 23131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:52.984726 23131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:52.987746 23131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:52.987962 23131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:52.995762 23131 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35935
I20260812 03:17:52.995836 23248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35935 every 8 connection(s)
I20260812 03:17:52.996821 23131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:17:52.999854 23249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:53.006016 23249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b: Bootstrap starting.
I20260812 03:17:53.006117 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 23131
I20260812 03:17:53.006281 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:17:53.006714 17253 external_mini_cluster.cc:1428] Setting key bba33fff53986358b8084a4bf19da906
I20260812 03:17:53.009037 23249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:53.009801 23249 log.cc:824] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:53.012028 23249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b: No bootstrap required, opened a new log
I20260812 03:17:53.016530 23249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d21f19809304b44ac367f8f2ef6df2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35935 } }
I20260812 03:17:53.016816 23249 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:53.016868 23249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d21f19809304b44ac367f8f2ef6df2b, State: Initialized, Role: FOLLOWER
I20260812 03:17:53.016963 23255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:53.008287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:53421 (local address 127.0.0.1:35935)
0812 03:17:53.008717 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:53.008733 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:53.008770 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:17:53.009586 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:53.009607 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:53.010361 (+ 754us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:53.010773 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:53.011105 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:53.012056 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:53.012984 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:53.013453 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:53.013630 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:53.014816 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:53.014826 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:53.014848 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:53.015391 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:53.015463 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:53.015493 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:53.015650 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:53.015822 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 03:17:53.015964 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":134,"threads_started":1}
I20260812 03:17:53.017424 23249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d21f19809304b44ac367f8f2ef6df2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35935 } }
I20260812 03:17:53.017591 23249 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:17:53.017702 23249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:17:53.017808 23249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:53.019984 23249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d21f19809304b44ac367f8f2ef6df2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35935 } }
I20260812 03:17:53.020386 23249 leader_election.cc:302] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d21f19809304b44ac367f8f2ef6df2b; no voters:
I20260812 03:17:53.020704 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:17:53.020769 23249 leader_election.cc:288] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:17:53.020838 23258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:53.021054 23258 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b [term 1 LEADER]: Becoming Leader. State: Replica: 3d21f19809304b44ac367f8f2ef6df2b, State: Running, Role: LEADER
I20260812 03:17:53.021413 23258 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d21f19809304b44ac367f8f2ef6df2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35935 } }
I20260812 03:17:53.022115 23249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:17:53.023106 23259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d21f19809304b44ac367f8f2ef6df2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d21f19809304b44ac367f8f2ef6df2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35935 } } }
I20260812 03:17:53.023252 23259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:53.023480 23259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d21f19809304b44ac367f8f2ef6df2b. Latest consensus state: current_term: 1 leader_uuid: "3d21f19809304b44ac367f8f2ef6df2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d21f19809304b44ac367f8f2ef6df2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35935 } } }
I20260812 03:17:53.023562 23259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:53.024573 23290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:17:53.028029 23290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:17:53.028319 23255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:53.023358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:46767 (local address 127.0.0.1:35935)
0812 03:17:53.023472 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:53.023476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:53.023715 (+ 239us) server_negotiation.cc:407] Connection header received
0812 03:17:53.023901 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:53.023905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:53.023955 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:53.024009 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:53.024488 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:53.025227 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:53.026369 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:53.026707 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:53.026832 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:53.027790 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:53.027793 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:53.027796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:53.027810 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:53.027840 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:53.027854 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:53.027948 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:53.028155 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 03:17:53.028194 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:17:53.033986 23290 catalog_manager.cc:1347] Generated new cluster ID: 4006c89d178c43e785c941affcbf30cf
I20260812 03:17:53.034111 23290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:17:53.060856 23290 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:17:53.062052 23290 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:17:53.069653 23290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d21f19809304b44ac367f8f2ef6df2b: Generated new TSK 0
I20260812 03:17:53.070514 23290 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:17:53.078373 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 23131
W20260812 03:17:53.104079 23097 connection.cc:535] client connection to 127.0.0.1:35935 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35935): Connection reset by peer (error 104)
2026-08-12T03:17:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1776 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:17:53 dist-test-slave-7nm7 krb5kdc[23315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:17:53 dist-test-slave-7nm7 krb5kdc[23315](info): set up 2 sockets
Aug 12 03:17:53 dist-test-slave-7nm7 krb5kdc[23315](info): commencing operation
krb5kdc: starting...
W20260812 03:17:54.402952 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:17:54 dist-test-slave-7nm7 krb5kdc[23315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:17:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:17:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:17:54.543413 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34255
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:54.663136 23338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:54.663466 23338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:54.663554 23338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:54.667584 23338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:54.667685 23338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:54.667723 23338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:54.667763 23338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:54.667801 23338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:54.673216 23338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:54.674396 23338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:54.675465 23338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:54.684443 23350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:54.684428 23347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:54.684434 23353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:54.685388 23338 server_base.cc:1034] running on GCE node
I20260812 03:17:54.685999 23338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:54.687247 23338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:54.688426 23338 hybrid_clock.cc:648] HybridClock initialized: now 1786504674688405 us; error 34 us; skew 500 ppm
Aug 12 03:17:54 dist-test-slave-7nm7 krb5kdc[23315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504674, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:54.691677 23338 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:17:54.692708 23338 webserver.cc:460] Webserver started at http://127.0.0.1:40201/ using document root <none> and password file <none>
I20260812 03:17:54.693315 23338 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:54.693380 23338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:54.693611 23338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:54.695819 23338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "2a59e4868bf14754a25d8d348f233eb4"
format_stamp: "Formatted at 2026-08-12 03:17:54 on dist-test-slave-7nm7"
server_key: "49b0b1a5b720afd36357507bad674550"
server_key_iv: "7197813a44c462960e5a26dba6115d4e"
server_key_version: "encryptionkey@0"
I20260812 03:17:54.696447 23338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "2a59e4868bf14754a25d8d348f233eb4"
format_stamp: "Formatted at 2026-08-12 03:17:54 on dist-test-slave-7nm7"
server_key: "49b0b1a5b720afd36357507bad674550"
server_key_iv: "7197813a44c462960e5a26dba6115d4e"
server_key_version: "encryptionkey@0"
I20260812 03:17:54.700634 23338 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:17:54.703276 23362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:54.704497 23338 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:17:54.704644 23338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "2a59e4868bf14754a25d8d348f233eb4"
format_stamp: "Formatted at 2026-08-12 03:17:54 on dist-test-slave-7nm7"
server_key: "49b0b1a5b720afd36357507bad674550"
server_key_iv: "7197813a44c462960e5a26dba6115d4e"
server_key_version: "encryptionkey@0"
I20260812 03:17:54.704772 23338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:54.709522 23338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:54.713109 23338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:54.713328 23338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:54.721465 23338 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34255
I20260812 03:17:54.721504 23459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34255 every 8 connection(s)
I20260812 03:17:54.722466 23338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:17:54.725401 23460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:54.730906 23460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4: Bootstrap starting.
I20260812 03:17:54.731509 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 23338
I20260812 03:17:54.731662 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:17:54.731961 17253 external_mini_cluster.cc:1428] Setting key 639a9b8f9d0a85f9497d7a51874d6f7a
I20260812 03:17:54.734407 23460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:54.735224 23460 log.cc:824] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:54.737598 23460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4: No bootstrap required, opened a new log
I20260812 03:17:54.737914 23463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:54.733628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:46081 (local address 127.0.0.1:34255)
0812 03:17:54.734144 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:54.734160 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:54.734195 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:17:54.734992 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:54.735034 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:54.735361 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:54.735703 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:54.736500 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:54.736506 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:54.736521 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:54.736534 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:54.736594 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:54.736620 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:54.736731 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:54.736825 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 03:17:54.736985 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":139,"threads_started":1}
I20260812 03:17:54.741137 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:17:54.742125 23460 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a59e4868bf14754a25d8d348f233eb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34255 } }
I20260812 03:17:54.742453 23460 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:54.742519 23460 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a59e4868bf14754a25d8d348f233eb4, State: Initialized, Role: FOLLOWER
I20260812 03:17:54.743027 23460 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a59e4868bf14754a25d8d348f233eb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34255 } }
I20260812 03:17:54.743193 23460 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:17:54.743278 23460 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:17:54.743403 23460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:54.744619 23460 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a59e4868bf14754a25d8d348f233eb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34255 } }
I20260812 03:17:54.745065 23460 leader_election.cc:302] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a59e4868bf14754a25d8d348f233eb4; no voters:
I20260812 03:17:54.745467 23460 leader_election.cc:288] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:17:54.745625 23463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:54.743436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:41177 (local address 127.0.0.1:34255)
0812 03:17:54.743586 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:54.743593 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:54.743908 (+ 315us) server_negotiation.cc:407] Connection header received
0812 03:17:54.743994 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:54.743997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:54.744036 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:54.744132 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:54.744989 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:54.744993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:54.744996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:54.745009 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:54.745041 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:54.745054 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:54.745164 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:54.745443 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 03:17:54.745493 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:17:54.745695 23475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:54.745916 23475 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4 [term 1 LEADER]: Becoming Leader. State: Replica: 2a59e4868bf14754a25d8d348f233eb4, State: Running, Role: LEADER
I20260812 03:17:54.746276 23475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a59e4868bf14754a25d8d348f233eb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34255 } }
I20260812 03:17:54.747038 23460 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:17:54.748036 23477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a59e4868bf14754a25d8d348f233eb4. Latest consensus state: current_term: 1 leader_uuid: "2a59e4868bf14754a25d8d348f233eb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a59e4868bf14754a25d8d348f233eb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34255 } } }
I20260812 03:17:54.748036 23476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a59e4868bf14754a25d8d348f233eb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a59e4868bf14754a25d8d348f233eb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34255 } } }
I20260812 03:17:54.748274 23476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:54.748275 23477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:54.748708 23484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:17:54.752583 23484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:17:54.758915 23484 catalog_manager.cc:1347] Generated new cluster ID: bec0e8aa185e49619233445dbaa52048
I20260812 03:17:54.759016 23484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:17:54.777798 23484 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:17:54.778177 23484 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:17:54.784660 23484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a59e4868bf14754a25d8d348f233eb4: Generated new TSK 0
I20260812 03:17:54.785358 23484 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:17:54.790940 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 23338
2026-08-12T03:17:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1700 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:17:54 dist-test-slave-7nm7 krb5kdc[23504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:17:54 dist-test-slave-7nm7 krb5kdc[23504](info): set up 2 sockets
Aug 12 03:17:54 dist-test-slave-7nm7 krb5kdc[23504](info): commencing operation
krb5kdc: starting...
W20260812 03:17:56.112406 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:17:56 dist-test-slave-7nm7 krb5kdc[23504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:17:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:17:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:17:56.255483 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46277
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:56.375149 23524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:56.375454 23524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:56.375501 23524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:56.379554 23524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:56.379635 23524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:56.379658 23524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:56.379678 23524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:56.379696 23524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:56.385174 23524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46277
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:56.386411 23524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:56.387454 23524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:56.396512 23546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:56.396533 23535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:56.396727 23524 server_base.cc:1034] running on GCE node
W20260812 03:17:56.396719 23548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:56.397516 23524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:56.398761 23524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:56.400130 23524 hybrid_clock.cc:648] HybridClock initialized: now 1786504676400059 us; error 70 us; skew 500 ppm
Aug 12 03:17:56 dist-test-slave-7nm7 krb5kdc[23504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504676, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:56.404575 23524 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:17:56.405989 23524 webserver.cc:460] Webserver started at http://127.0.0.1:32953/ using document root <none> and password file <none>
I20260812 03:17:56.406746 23524 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:56.406850 23524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:56.407114 23524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:56.409313 23524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "936f95c5e79f485795b26096264729a0"
format_stamp: "Formatted at 2026-08-12 03:17:56 on dist-test-slave-7nm7"
server_key: "a22393b87e1a4752234d04a8a26637bb"
server_key_iv: "fea9936935623db18af2e87239f82b6f"
server_key_version: "encryptionkey@0"
I20260812 03:17:56.409915 23524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "936f95c5e79f485795b26096264729a0"
format_stamp: "Formatted at 2026-08-12 03:17:56 on dist-test-slave-7nm7"
server_key: "a22393b87e1a4752234d04a8a26637bb"
server_key_iv: "fea9936935623db18af2e87239f82b6f"
server_key_version: "encryptionkey@0"
I20260812 03:17:56.414477 23524 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:17:56.417412 23567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:56.418820 23524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:17:56.418985 23524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "936f95c5e79f485795b26096264729a0"
format_stamp: "Formatted at 2026-08-12 03:17:56 on dist-test-slave-7nm7"
server_key: "a22393b87e1a4752234d04a8a26637bb"
server_key_iv: "fea9936935623db18af2e87239f82b6f"
server_key_version: "encryptionkey@0"
I20260812 03:17:56.419298 23524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:56.455864 23524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:56.459332 23524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:56.459571 23524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:56.467881 23524 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46277
I20260812 03:17:56.467924 23648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46277 every 8 connection(s)
I20260812 03:17:56.469015 23524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:17:56.471959 23650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:56.474692 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 23524
I20260812 03:17:56.474870 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:17:56.475198 17253 external_mini_cluster.cc:1428] Setting key 8809b99254306d7809672e82884c1d91
I20260812 03:17:56.478089 23650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0: Bootstrap starting.
I20260812 03:17:56.480911 23650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:56.481758 23650 log.cc:824] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:56.483934 23650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0: No bootstrap required, opened a new log
Aug 12 03:17:56 dist-test-slave-7nm7 krb5kdc[23504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:17:56.488335 23650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "936f95c5e79f485795b26096264729a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46277 } }
I20260812 03:17:56.488633 23650 raft_consensus.cc:374] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:56.488716 23650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 936f95c5e79f485795b26096264729a0, State: Initialized, Role: FOLLOWER
I20260812 03:17:56.489274 23650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "936f95c5e79f485795b26096264729a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46277 } }
I20260812 03:17:56.489439 23650 raft_consensus.cc:388] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:17:56.489521 23650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:17:56.489624 23650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:56.490912 23650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "936f95c5e79f485795b26096264729a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46277 } }
I20260812 03:17:56.491324 23650 leader_election.cc:302] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 936f95c5e79f485795b26096264729a0; no voters:
I20260812 03:17:56.491698 23650 leader_election.cc:288] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:17:56.491794 23661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:56.492053 23661 raft_consensus.cc:686] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0 [term 1 LEADER]: Becoming Leader. State: Replica: 936f95c5e79f485795b26096264729a0, State: Running, Role: LEADER
I20260812 03:17:56.492377 23655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:56.476703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:51151 (local address 127.0.0.1:46277)
0812 03:17:56.477157 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:56.477176 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:56.477221 (+ 45us) server_negotiation.cc:407] Connection header received
0812 03:17:56.478004 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:56.478026 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:56.478507 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:56.478855 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:56.479544 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:56.481124 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:56.482036 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:56.482664 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:56.482841 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:56.485802 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:56.485809 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:56.485826 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:56.485860 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:56.488596 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:56.489216 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:56.489222 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:56.489228 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:56.489371 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:56.489876 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:56.489883 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:56.489886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:56.490442 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:56.490667 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:56.491015 (+ 348us) server_negotiation.cc:299] Negotiation successful
0812 03:17:56.491304 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":112,"threads_started":1}
I20260812 03:17:56.492409 23661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "936f95c5e79f485795b26096264729a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46277 } }
I20260812 03:17:56.493012 23650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:17:56.494216 23662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "936f95c5e79f485795b26096264729a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "936f95c5e79f485795b26096264729a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46277 } } }
I20260812 03:17:56.494390 23662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:56.494757 23663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 936f95c5e79f485795b26096264729a0. Latest consensus state: current_term: 1 leader_uuid: "936f95c5e79f485795b26096264729a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "936f95c5e79f485795b26096264729a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46277 } } }
I20260812 03:17:56.494861 23663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:56.496152 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 03:17:56.498567 23684 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:17:56.498694 23684 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:17:56.498836 23675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:17:56.499533 23675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:17:56.506158 23675 catalog_manager.cc:1347] Generated new cluster ID: 46e24ffbd2914b909893439cf21292ec
I20260812 03:17:56.506278 23675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:17:56.507992 23655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:56.498804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:39565 (local address 127.0.0.1:46277)
0812 03:17:56.498926 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:56.498931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:56.499238 (+ 307us) server_negotiation.cc:407] Connection header received
0812 03:17:56.499405 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:56.499408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:56.499450 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:56.499535 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:56.500242 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:56.500757 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:56.501566 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:56.501814 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:56.501929 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:56.503674 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:56.503678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:56.503681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:17:56.503711 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:56.505692 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:56.506400 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:56.506404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:56.506406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:56.506500 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:17:56.506950 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:17:56.506953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:17:56.506956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:17:56.507206 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:56.507377 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:56.507711 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 03:17:56.507818 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:17:56.524814 23675 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:17:56.525769 23675 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:17:56.531658 23675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 936f95c5e79f485795b26096264729a0: Generated new TSK 0
I20260812 03:17:56.532402 23675 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:17:56.539583 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 23524
W20260812 03:17:56.563405 23497 connection.cc:535] client connection to 127.0.0.1:46277 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46277): Connection reset by peer (error 104)
2026-08-12T03:17:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1758 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:17:56 dist-test-slave-7nm7 krb5kdc[23702](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:17:56 dist-test-slave-7nm7 krb5kdc[23702](info): set up 2 sockets
Aug 12 03:17:56 dist-test-slave-7nm7 krb5kdc[23702](info): commencing operation
krb5kdc: starting...
W20260812 03:17:57.899114 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:17:57 dist-test-slave-7nm7 krb5kdc[23702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:17:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:17:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:17:58.035694 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42149
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:58.153471 23724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:58.153785 23724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:58.153877 23724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:58.157977 23724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:58.158072 23724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:58.158113 23724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:58.158182 23724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:58.158223 23724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:58.163874 23724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42149
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:58.165096 23724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:58.166154 23724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:58.175763 23731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:58.175763 23737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:58.175838 23733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:58.176204 23724 server_base.cc:1034] running on GCE node
I20260812 03:17:58.176719 23724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:58.177831 23724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:58.179023 23724 hybrid_clock.cc:648] HybridClock initialized: now 1786504678179015 us; error 48 us; skew 500 ppm
Aug 12 03:17:58 dist-test-slave-7nm7 krb5kdc[23702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504678, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:58.186182 23724 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:17:58.187296 23724 webserver.cc:460] Webserver started at http://127.0.0.1:37031/ using document root <none> and password file <none>
I20260812 03:17:58.187923 23724 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:58.188035 23724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:58.188300 23724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:58.190590 23724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "38480ce9835c4023b3b2ace2968f3d4c"
format_stamp: "Formatted at 2026-08-12 03:17:58 on dist-test-slave-7nm7"
server_key: "4ddd65465ec3bfb52177dd43bcb90155"
server_key_iv: "67c4a98aa00626a73f617eeed3d9c045"
server_key_version: "encryptionkey@0"
I20260812 03:17:58.191218 23724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "38480ce9835c4023b3b2ace2968f3d4c"
format_stamp: "Formatted at 2026-08-12 03:17:58 on dist-test-slave-7nm7"
server_key: "4ddd65465ec3bfb52177dd43bcb90155"
server_key_iv: "67c4a98aa00626a73f617eeed3d9c045"
server_key_version: "encryptionkey@0"
I20260812 03:17:58.195575 23724 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:17:58.198066 23752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:58.199379 23724 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 03:17:58.199545 23724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "38480ce9835c4023b3b2ace2968f3d4c"
format_stamp: "Formatted at 2026-08-12 03:17:58 on dist-test-slave-7nm7"
server_key: "4ddd65465ec3bfb52177dd43bcb90155"
server_key_iv: "67c4a98aa00626a73f617eeed3d9c045"
server_key_version: "encryptionkey@0"
I20260812 03:17:58.199672 23724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:58.251320 23724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:58.256023 23724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:58.256316 23724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:58.265269 23724 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42149
I20260812 03:17:58.265296 23838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42149 every 8 connection(s)
I20260812 03:17:58.266492 23724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:17:58.270255 23839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:17:58.276062 23839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c: Bootstrap starting.
I20260812 03:17:58.276618 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 23724
I20260812 03:17:58.276796 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:17:58.277069 17253 external_mini_cluster.cc:1428] Setting key 67f74f6c74e9959f0b5df76996932b7f
I20260812 03:17:58.279080 23839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c: Neither blocks nor log segments found. Creating new log.
I20260812 03:17:58.279815 23839 log.cc:824] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:17:58.281868 23839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c: No bootstrap required, opened a new log
I20260812 03:17:58.286296 23839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38480ce9835c4023b3b2ace2968f3d4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42149 } }
I20260812 03:17:58.286592 23839 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:17:58.286684 23839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38480ce9835c4023b3b2ace2968f3d4c, State: Initialized, Role: FOLLOWER
I20260812 03:17:58.287020 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:58.278556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:35299 (local address 127.0.0.1:42149)
0812 03:17:58.279006 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:58.279022 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:58.279056 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:17:58.279769 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:58.279786 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:58.280206 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:58.280582 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:58.280966 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:58.281932 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:58.283230 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:58.283802 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:58.283954 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:58.284930 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:58.284935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:58.284948 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:58.285355 (+ 407us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:58.285412 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:58.285436 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:58.285537 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:58.285798 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 03:17:58.285952 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":137,"threads_started":1}
I20260812 03:17:58.287181 23839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38480ce9835c4023b3b2ace2968f3d4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42149 } }
I20260812 03:17:58.287344 23839 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:17:58.287416 23839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:17:58.287513 23839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:17:58.288574 23839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38480ce9835c4023b3b2ace2968f3d4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42149 } }
I20260812 03:17:58.288936 23839 leader_election.cc:302] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38480ce9835c4023b3b2ace2968f3d4c; no voters:
I20260812 03:17:58.289309 23839 leader_election.cc:288] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:17:58.289379 23851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:17:58.289597 23851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c [term 1 LEADER]: Becoming Leader. State: Replica: 38480ce9835c4023b3b2ace2968f3d4c, State: Running, Role: LEADER
I20260812 03:17:58.289912 23851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38480ce9835c4023b3b2ace2968f3d4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42149 } }
I20260812 03:17:58.290063 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:17:58.290724 23839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:17:58.291742 23852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38480ce9835c4023b3b2ace2968f3d4c. Latest consensus state: current_term: 1 leader_uuid: "38480ce9835c4023b3b2ace2968f3d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38480ce9835c4023b3b2ace2968f3d4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42149 } } }
I20260812 03:17:58.291761 23853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38480ce9835c4023b3b2ace2968f3d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38480ce9835c4023b3b2ace2968f3d4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42149 } } }
I20260812 03:17:58.291925 23853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c [sys.catalog]: This master's current role is: LEADER
I20260812 03:17:58.291917 23852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c [sys.catalog]: This master's current role is: LEADER
W20260812 03:17:58.296015 23888 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:17:58.296475 23888 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:17:58.296558 23873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:17:58.297531 23873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:17:58.298175 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:17:58.292662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:60083 (local address 127.0.0.1:42149)
0812 03:17:58.292781 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 03:17:58.292786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:17:58.293499 (+ 713us) server_negotiation.cc:407] Connection header received
0812 03:17:58.293618 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:17:58.293621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:17:58.293673 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:17:58.293779 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:17:58.294768 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:58.295324 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:58.296249 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:17:58.296600 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:17:58.296733 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:17:58.297655 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:17:58.297659 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:17:58.297661 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:17:58.297675 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:17:58.297714 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:17:58.297724 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:17:58.297823 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:17:58.297979 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 03:17:58.298017 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:17:58.303894 23873 catalog_manager.cc:1347] Generated new cluster ID: 742d32eb41b440f6b59f630406ea0ab4
I20260812 03:17:58.303975 23873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:17:58.314600 23873 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:17:58.315714 23873 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:17:58.326000 23873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38480ce9835c4023b3b2ace2968f3d4c: Generated new TSK 0
I20260812 03:17:58.326799 23873 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:17:58.337057 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 23724
W20260812 03:17:58.353310 23700 connection.cc:535] client connection to 127.0.0.1:42149 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42149): Connection reset by peer (error 104)
2026-08-12T03:17:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1789 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:17:58 dist-test-slave-7nm7 krb5kdc[23897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:17:58 dist-test-slave-7nm7 krb5kdc[23897](info): set up 2 sockets
Aug 12 03:17:58 dist-test-slave-7nm7 krb5kdc[23897](info): commencing operation
krb5kdc: starting...
W20260812 03:17:59.685608 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.317s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:17:59 dist-test-slave-7nm7 krb5kdc[23897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:17:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:17:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:17:59.819859 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41979
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:17:59.937570 23923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:17:59.937868 23923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:17:59.937916 23923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:17:59.941854 23923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:17:59.941920 23923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:17:59.941943 23923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:17:59.941989 23923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:17:59.942008 23923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:17:59.947386 23923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41979
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:17:59.948590 23923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:17:59.949700 23923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:17:59.959121 23943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:59.959113 23937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:17:59.959113 23940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:17:59.959771 23923 server_base.cc:1034] running on GCE node
I20260812 03:17:59.960486 23923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:17:59.961825 23923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:17:59.963086 23923 hybrid_clock.cc:648] HybridClock initialized: now 1786504679963072 us; error 43 us; skew 500 ppm
Aug 12 03:17:59 dist-test-slave-7nm7 krb5kdc[23897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504679, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:17:59.966435 23923 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:17:59.967684 23923 webserver.cc:460] Webserver started at http://127.0.0.1:41221/ using document root <none> and password file <none>
I20260812 03:17:59.968585 23923 fs_manager.cc:359] Metadata directory not provided
I20260812 03:17:59.968698 23923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:17:59.968967 23923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:17:59.971365 23923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "3ce57ae58007422c81f79b3a0590e8c5"
format_stamp: "Formatted at 2026-08-12 03:17:59 on dist-test-slave-7nm7"
server_key: "ae6e203b7823747bb8d677c1e97ae171"
server_key_iv: "107e5cf0a341e3ffac3ca2ed6ca97dce"
server_key_version: "encryptionkey@0"
I20260812 03:17:59.972002 23923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "3ce57ae58007422c81f79b3a0590e8c5"
format_stamp: "Formatted at 2026-08-12 03:17:59 on dist-test-slave-7nm7"
server_key: "ae6e203b7823747bb8d677c1e97ae171"
server_key_iv: "107e5cf0a341e3ffac3ca2ed6ca97dce"
server_key_version: "encryptionkey@0"
I20260812 03:17:59.976393 23923 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:17:59.978973 23954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:17:59.980300 23923 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:17:59.980463 23923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "3ce57ae58007422c81f79b3a0590e8c5"
format_stamp: "Formatted at 2026-08-12 03:17:59 on dist-test-slave-7nm7"
server_key: "ae6e203b7823747bb8d677c1e97ae171"
server_key_iv: "107e5cf0a341e3ffac3ca2ed6ca97dce"
server_key_version: "encryptionkey@0"
I20260812 03:17:59.980585 23923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:17:59.985688 23923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:17:59.989292 23923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:17:59.989534 23923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:17:59.997624 23923 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41979
I20260812 03:17:59.997653 24044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41979 every 8 connection(s)
I20260812 03:17:59.998713 23923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:18:00.002110 24047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:18:00.007850 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 23923
I20260812 03:18:00.008028 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:18:00.008170 24047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5: Bootstrap starting.
I20260812 03:18:00.008358 17253 external_mini_cluster.cc:1428] Setting key 84440a1152095e5192fc5debc350cb5b
I20260812 03:18:00.010895 24047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5: Neither blocks nor log segments found. Creating new log.
I20260812 03:18:00.011751 24047 log.cc:824] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5: Log is configured to *not* fsync() on all Append() calls
I20260812 03:18:00.013640 24051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:00.009715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:41319 (local address 127.0.0.1:41979)
0812 03:18:00.010163 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:00.010178 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:00.010213 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:18:00.010906 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:00.010922 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:00.011264 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:00.011532 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:00.012387 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:00.012393 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:00.012413 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:18:00.012425 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:00.012478 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:18:00.012498 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:00.012622 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:00.012727 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 03:18:00.012892 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":149,"threads_started":1}
I20260812 03:18:00.013974 24047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5: No bootstrap required, opened a new log
I20260812 03:18:00.016649 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:18:00.018182 24047 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce57ae58007422c81f79b3a0590e8c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41979 } }
I20260812 03:18:00.018471 24047 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:18:00.018529 24047 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce57ae58007422c81f79b3a0590e8c5, State: Initialized, Role: FOLLOWER
I20260812 03:18:00.018982 24047 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce57ae58007422c81f79b3a0590e8c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41979 } }
I20260812 03:18:00.019170 24047 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:18:00.019258 24047 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:18:00.019366 24047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:18:00.020243 24047 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce57ae58007422c81f79b3a0590e8c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41979 } }
I20260812 03:18:00.020515 24051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:00.018771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:44865 (local address 127.0.0.1:41979)
0812 03:18:00.018913 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:00.018917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:00.019158 (+ 241us) server_negotiation.cc:407] Connection header received
0812 03:18:00.019260 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:00.019264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:00.019297 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:00.019383 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:00.020113 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:00.020116 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:00.020120 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:18:00.020134 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:00.020161 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:18:00.020173 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:00.020272 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:00.020358 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 03:18:00.020399 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:18:00.020628 24047 leader_election.cc:302] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ce57ae58007422c81f79b3a0590e8c5; no voters:
I20260812 03:18:00.021032 24047 leader_election.cc:288] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:18:00.021117 24063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:18:00.021435 24063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce57ae58007422c81f79b3a0590e8c5, State: Running, Role: LEADER
I20260812 03:18:00.021786 24063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce57ae58007422c81f79b3a0590e8c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41979 } }
I20260812 03:18:00.022444 24047 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:18:00.023619 24067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ce57ae58007422c81f79b3a0590e8c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce57ae58007422c81f79b3a0590e8c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41979 } } }
I20260812 03:18:00.023625 24064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ce57ae58007422c81f79b3a0590e8c5. Latest consensus state: current_term: 1 leader_uuid: "3ce57ae58007422c81f79b3a0590e8c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce57ae58007422c81f79b3a0590e8c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41979 } } }
I20260812 03:18:00.023841 24064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:00.023833 24067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:00.024238 24082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:18:00.027338 24082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:18:00.032459 24082 catalog_manager.cc:1347] Generated new cluster ID: c45c41c70c5b47e18fcf109352281ece
I20260812 03:18:00.032536 24082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:18:00.043001 24082 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:18:00.043320 24082 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:18:00.060292 24082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ce57ae58007422c81f79b3a0590e8c5: Generated new TSK 0
I20260812 03:18:00.061129 24082 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:18:00.072325 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 23923
2026-08-12T03:18:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1736 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:18:00 dist-test-slave-7nm7 krb5kdc[24107](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:18:00 dist-test-slave-7nm7 krb5kdc[24107](info): set up 2 sockets
Aug 12 03:18:00 dist-test-slave-7nm7 krb5kdc[24107](info): commencing operation
krb5kdc: starting...
W20260812 03:18:01.422557 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.318s user 0.004s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:18:01 dist-test-slave-7nm7 krb5kdc[24107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:18:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:18:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:18:01.558140 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:33381
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:33381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:18:01.697674 24129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:18:01.698549 24129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:18:01.698730 24129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:18:01.705571 24129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:18:01.705677 24129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:18:01.705700 24129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:18:01.705721 24129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:18:01.705811 24129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:18:01.712272 24129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:33381
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:33381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:18:01.713574 24129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:18:01.714742 24129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:18:01.725371 24139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:18:01.725375 24134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:18:01.725371 24135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:18:01.725745 24129 server_base.cc:1034] running on GCE node
I20260812 03:18:01.726464 24129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:18:01.727715 24129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:18:01.728945 24129 hybrid_clock.cc:648] HybridClock initialized: now 1786504681728927 us; error 38 us; skew 500 ppm
Aug 12 03:18:01 dist-test-slave-7nm7 krb5kdc[24107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504681, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:18:01.732039 24129 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:18:01.733093 24129 webserver.cc:460] Webserver started at http://127.16.217.126:33699/ using document root <none> and password file <none>
I20260812 03:18:01.733659 24129 fs_manager.cc:359] Metadata directory not provided
I20260812 03:18:01.733723 24129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:18:01.733898 24129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:18:01.737435 24129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "a77e4df308d742d3ae39aa86e502aef2"
format_stamp: "Formatted at 2026-08-12 03:18:01 on dist-test-slave-7nm7"
server_key: "2d689a34623fc9e7beb317dbdecd6b0f"
server_key_iv: "ee3c0229410aa5a1484126ed20f322ec"
server_key_version: "encryptionkey@0"
I20260812 03:18:01.738109 24129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "a77e4df308d742d3ae39aa86e502aef2"
format_stamp: "Formatted at 2026-08-12 03:18:01 on dist-test-slave-7nm7"
server_key: "2d689a34623fc9e7beb317dbdecd6b0f"
server_key_iv: "ee3c0229410aa5a1484126ed20f322ec"
server_key_version: "encryptionkey@0"
I20260812 03:18:01.742899 24129 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:18:01.745724 24149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:18:01.747313 24129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:18:01.747443 24129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "a77e4df308d742d3ae39aa86e502aef2"
format_stamp: "Formatted at 2026-08-12 03:18:01 on dist-test-slave-7nm7"
server_key: "2d689a34623fc9e7beb317dbdecd6b0f"
server_key_iv: "ee3c0229410aa5a1484126ed20f322ec"
server_key_version: "encryptionkey@0"
I20260812 03:18:01.747610 24129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:18:01.785880 24129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:18:01.789594 24129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:18:01.789830 24129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:18:01.797894 24129 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:33381
I20260812 03:18:01.797922 24226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:33381 every 8 connection(s)
I20260812 03:18:01.799031 24129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:18:01.801885 24230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:18:01.807639 24230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2: Bootstrap starting.
I20260812 03:18:01.808483 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 24129
I20260812 03:18:01.808636 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:18:01.808995 17253 external_mini_cluster.cc:1428] Setting key 0742b01e4815e3cd94993df1f4e74125
I20260812 03:18:01.810976 24230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2: Neither blocks nor log segments found. Creating new log.
I20260812 03:18:01.811753 24230 log.cc:824] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2: Log is configured to *not* fsync() on all Append() calls
I20260812 03:18:01.813747 24230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2: No bootstrap required, opened a new log
I20260812 03:18:01.817379 24230 raft_consensus.cc:348] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a77e4df308d742d3ae39aa86e502aef2" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 33381 } }
I20260812 03:18:01.817648 24230 raft_consensus.cc:374] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:18:01.817741 24230 raft_consensus.cc:729] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a77e4df308d742d3ae39aa86e502aef2, State: Initialized, Role: FOLLOWER
I20260812 03:18:01.818343 24230 consensus_queue.cc:260] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a77e4df308d742d3ae39aa86e502aef2" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 33381 } }
I20260812 03:18:01.818514 24230 raft_consensus.cc:388] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:18:01.818593 24230 raft_consensus.cc:482] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:18:01.818708 24230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 03:18:01 dist-test-slave-7nm7 krb5kdc[24107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:18:01.820072 24230 raft_consensus.cc:504] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a77e4df308d742d3ae39aa86e502aef2" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 33381 } }
I20260812 03:18:01.820495 24230 leader_election.cc:302] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a77e4df308d742d3ae39aa86e502aef2; no voters:
I20260812 03:18:01.820941 24230 leader_election.cc:288] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:18:01.821048 24240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:18:01.821611 24240 raft_consensus.cc:686] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2 [term 1 LEADER]: Becoming Leader. State: Replica: a77e4df308d742d3ae39aa86e502aef2, State: Running, Role: LEADER
I20260812 03:18:01.822005 24240 consensus_queue.cc:237] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a77e4df308d742d3ae39aa86e502aef2" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 33381 } }
I20260812 03:18:01.822448 24230 sys_catalog.cc:564] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:18:01.823902 24241 sys_catalog.cc:455] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a77e4df308d742d3ae39aa86e502aef2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a77e4df308d742d3ae39aa86e502aef2" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 33381 } } }
I20260812 03:18:01.824056 24241 sys_catalog.cc:458] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2 [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:01.824445 24242 sys_catalog.cc:455] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a77e4df308d742d3ae39aa86e502aef2. Latest consensus state: current_term: 1 leader_uuid: "a77e4df308d742d3ae39aa86e502aef2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a77e4df308d742d3ae39aa86e502aef2" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 33381 } } }
I20260812 03:18:01.824602 24242 sys_catalog.cc:458] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2 [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:01.827459 24234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:01.810393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39515 (local address 127.16.217.126:33381)
0812 03:18:01.810865 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:01.810885 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:01.810922 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:18:01.811633 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:01.811653 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:01.812057 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:01.812404 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:01.813223 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:01.814315 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:01.815164 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:01.816352 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:01.816561 (+ 209us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:01.820088 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:01.820094 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:01.820108 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:01.820140 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:01.822296 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:01.822987 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:01.822993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:01.822999 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:01.823134 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:01.823586 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:01.823589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:01.823591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:01.824251 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:01.826007 (+ 1756us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:01.826178 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 03:18:01.826543 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":149,"threads_started":1}
W20260812 03:18:01.828290 24272 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:18:01.828401 24272 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:18:01.828506 24257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:18:01.829255 24257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:18:01.830859 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:18:01.835253 24257 catalog_manager.cc:1347] Generated new cluster ID: e8246bf178ba4261a81e5c0b91ba4b90
I20260812 03:18:01.835374 24257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:18:01.843717 24234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:01.833403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51849 (local address 127.16.217.126:33381)
0812 03:18:01.833533 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:01.833539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:01.833558 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:18:01.833713 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:01.833717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:01.833771 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:01.833874 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:01.834721 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:01.835527 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:01.836716 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:01.837098 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:01.837184 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:01.839312 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:01.839316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:01.839319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:01.839348 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:01.841192 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:01.841866 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:01.841871 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:01.841873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:01.842030 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:01.842540 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:01.842544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:01.842546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:01.842779 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:01.842914 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:01.843425 (+ 511us) server_negotiation.cc:299] Negotiation successful
0812 03:18:01.843538 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:18:01.855603 24257 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:18:01.856894 24257 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:18:01.873593 24257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a77e4df308d742d3ae39aa86e502aef2: Generated new TSK 0
I20260812 03:18:01.874579 24257 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:18:01.880358 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 24129
W20260812 03:18:01.893574 24102 connection.cc:535] client connection to 127.16.217.126:33381 recv error: Network error: failed to read from TLS socket (remote: 127.16.217.126:33381): Connection reset by peer (error 104)
2026-08-12T03:18:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1803 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:18:01 dist-test-slave-7nm7 krb5kdc[24291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:18:01 dist-test-slave-7nm7 krb5kdc[24291](info): set up 2 sockets
Aug 12 03:18:01 dist-test-slave-7nm7 krb5kdc[24291](info): commencing operation
krb5kdc: starting...
W20260812 03:18:03.198062 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:18:03 dist-test-slave-7nm7 krb5kdc[24291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:18:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:18:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:18:03.331717 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:37265
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:41735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:37265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:18:03.463037 24315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:18:03.463382 24315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:18:03.463490 24315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:18:03.467881 24315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:18:03.467991 24315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:18:03.468042 24315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:18:03.468106 24315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:18:03.468151 24315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:18:03.473707 24315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:41735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:37265
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:37265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:18:03.475059 24315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:18:03.476199 24315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:18:03.485527 24326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:18:03.485626 24315 server_base.cc:1034] running on GCE node
W20260812 03:18:03.485527 24325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:18:03.485500 24331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:18:03.486466 24315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:18:03.487612 24315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:18:03.488814 24315 hybrid_clock.cc:648] HybridClock initialized: now 1786504683488794 us; error 41 us; skew 500 ppm
Aug 12 03:18:03 dist-test-slave-7nm7 krb5kdc[24291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504683, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:18:03.499442 24315 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:18:03.500494 24315 webserver.cc:460] Webserver started at http://127.16.217.126:45349/ using document root <none> and password file <none>
I20260812 03:18:03.501106 24315 fs_manager.cc:359] Metadata directory not provided
I20260812 03:18:03.501222 24315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:18:03.501479 24315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:18:03.503751 24315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "9c1c441003134586a13389eb3be9f4c3"
format_stamp: "Formatted at 2026-08-12 03:18:03 on dist-test-slave-7nm7"
server_key: "d35ecc278ea61fb8f0286e8fe8c68e6d"
server_key_iv: "4a0991a3543609a2fafaadc7099ea788"
server_key_version: "encryptionkey@0"
I20260812 03:18:03.504356 24315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "9c1c441003134586a13389eb3be9f4c3"
format_stamp: "Formatted at 2026-08-12 03:18:03 on dist-test-slave-7nm7"
server_key: "d35ecc278ea61fb8f0286e8fe8c68e6d"
server_key_iv: "4a0991a3543609a2fafaadc7099ea788"
server_key_version: "encryptionkey@0"
I20260812 03:18:03.508363 24315 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:18:03.511121 24346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:18:03.512392 24315 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:18:03.512549 24315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "9c1c441003134586a13389eb3be9f4c3"
format_stamp: "Formatted at 2026-08-12 03:18:03 on dist-test-slave-7nm7"
server_key: "d35ecc278ea61fb8f0286e8fe8c68e6d"
server_key_iv: "4a0991a3543609a2fafaadc7099ea788"
server_key_version: "encryptionkey@0"
I20260812 03:18:03.512670 24315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:18:03.533486 24315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:18:03.536937 24315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:18:03.537195 24315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:18:03.546033 24315 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:37265
I20260812 03:18:03.546162 24429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:37265 every 8 connection(s)
I20260812 03:18:03.547180 24315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:18:03.550551 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 24315
I20260812 03:18:03.550581 24430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:18:03.550704 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:18:03.550980 17253 external_mini_cluster.cc:1428] Setting key f974e60da48c3592da0244a5c2eca447
I20260812 03:18:03.558359 24430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3: Bootstrap starting.
I20260812 03:18:03.561122 24433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:03.552373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60769 (local address 127.16.217.126:37265)
0812 03:18:03.552886 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:03.552901 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:03.552937 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:18:03.553582 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:03.553600 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:03.554049 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:03.554575 (+ 526us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:03.555014 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:03.556018 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:03.557299 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:03.557769 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:03.557893 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:03.558919 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:03.558926 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:03.558941 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:18:03.559420 (+ 479us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:03.559487 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:18:03.559521 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:03.559685 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:03.559854 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 03:18:03.559984 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":148,"threads_started":1}
I20260812 03:18:03.561385 24430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3: Neither blocks nor log segments found. Creating new log.
I20260812 03:18:03.562294 24430 log.cc:824] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:18:03.564808 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:18:03.564889 24430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3: No bootstrap required, opened a new log
I20260812 03:18:03.571830 24430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c1c441003134586a13389eb3be9f4c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 37265 } }
I20260812 03:18:03.572146 24430 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:18:03.572225 24430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c1c441003134586a13389eb3be9f4c3, State: Initialized, Role: FOLLOWER
I20260812 03:18:03.572745 24430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c1c441003134586a13389eb3be9f4c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 37265 } }
I20260812 03:18:03.572925 24430 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:18:03.573061 24433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:03.567709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43727 (local address 127.16.217.126:37265)
0812 03:18:03.567857 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:03.567863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:03.568004 (+ 141us) server_negotiation.cc:407] Connection header received
0812 03:18:03.568249 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:03.568335 (+ 86us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:03.568406 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:03.568520 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:03.568787 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:03.569892 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:03.570898 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:03.571274 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:03.571400 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:03.572504 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:03.572508 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:03.572510 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:18:03.572522 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:03.572545 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:18:03.572555 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:03.572665 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:03.572823 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 03:18:03.572860 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:18:03.573154 24430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:18:03.573338 24430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:18:03.574776 24430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c1c441003134586a13389eb3be9f4c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 37265 } }
I20260812 03:18:03.575151 24430 leader_election.cc:302] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c1c441003134586a13389eb3be9f4c3; no voters:
I20260812 03:18:03.575507 24430 leader_election.cc:288] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:18:03.575628 24445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:18:03.575894 24445 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3 [term 1 LEADER]: Becoming Leader. State: Replica: 9c1c441003134586a13389eb3be9f4c3, State: Running, Role: LEADER
I20260812 03:18:03.576259 24445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c1c441003134586a13389eb3be9f4c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 37265 } }
I20260812 03:18:03.576794 24430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:18:03.577970 24452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c1c441003134586a13389eb3be9f4c3. Latest consensus state: current_term: 1 leader_uuid: "9c1c441003134586a13389eb3be9f4c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c1c441003134586a13389eb3be9f4c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 37265 } } }
I20260812 03:18:03.577970 24451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c1c441003134586a13389eb3be9f4c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c1c441003134586a13389eb3be9f4c3" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 37265 } } }
I20260812 03:18:03.578157 24451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3 [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:03.578157 24452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3 [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:03.578715 24469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:18:03.582010 24469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:18:03.587201 24469 catalog_manager.cc:1347] Generated new cluster ID: 2183dc565f2f4cab95176b6cb9639ed8
I20260812 03:18:03.587263 24469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:18:03.599248 24469 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:18:03.600246 24469 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:18:03.608741 24469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c1c441003134586a13389eb3be9f4c3: Generated new TSK 0
I20260812 03:18:03.609445 24469 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:18:03.622478 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 24315
W20260812 03:18:03.631830 24289 connection.cc:535] client connection to 127.16.217.126:37265 recv error: Network error: failed to read from TLS socket (remote: 127.16.217.126:37265): Connection reset by peer (error 104)
2026-08-12T03:18:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1738 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:18:03 dist-test-slave-7nm7 krb5kdc[24490](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:18:03 dist-test-slave-7nm7 krb5kdc[24490](info): set up 2 sockets
Aug 12 03:18:03 dist-test-slave-7nm7 krb5kdc[24490](info): commencing operation
krb5kdc: starting...
W20260812 03:18:04.978175 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.331s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:18:05 dist-test-slave-7nm7 krb5kdc[24490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:18:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:18:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:18:05.110579 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:43095
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:37633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:43095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:18:05.232394 24524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:18:05.232712 24524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:18:05.232802 24524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:18:05.236816 24524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:18:05.236922 24524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:18:05.236974 24524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:18:05.237032 24524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:18:05.237076 24524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:18:05.242499 24524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:43095
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:43095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:18:05.243637 24524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:18:05.244736 24524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:18:05.253832 24541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:18:05.253803 24539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:18:05.253808 24536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:18:05.254207 24524 server_base.cc:1034] running on GCE node
I20260812 03:18:05.254832 24524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:18:05.255976 24524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:18:05.257191 24524 hybrid_clock.cc:648] HybridClock initialized: now 1786504685257171 us; error 34 us; skew 500 ppm
Aug 12 03:18:05 dist-test-slave-7nm7 krb5kdc[24490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504685, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:18:05.260265 24524 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:18:05.261294 24524 webserver.cc:460] Webserver started at http://127.16.217.126:34183/ using document root <none> and password file <none>
I20260812 03:18:05.261907 24524 fs_manager.cc:359] Metadata directory not provided
I20260812 03:18:05.262006 24524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:18:05.262439 24524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:18:05.264705 24524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "ec9a526c492b42659c97328aa9999b63"
format_stamp: "Formatted at 2026-08-12 03:18:05 on dist-test-slave-7nm7"
server_key: "d1b2c165106ab9e532792aea79b8b62c"
server_key_iv: "17c43f7e1ea21b3db43a853d176b74f5"
server_key_version: "encryptionkey@0"
I20260812 03:18:05.265372 24524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "ec9a526c492b42659c97328aa9999b63"
format_stamp: "Formatted at 2026-08-12 03:18:05 on dist-test-slave-7nm7"
server_key: "d1b2c165106ab9e532792aea79b8b62c"
server_key_iv: "17c43f7e1ea21b3db43a853d176b74f5"
server_key_version: "encryptionkey@0"
I20260812 03:18:05.269155 24524 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260812 03:18:05.271991 24550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:18:05.273438 24524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:18:05.273604 24524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "ec9a526c492b42659c97328aa9999b63"
format_stamp: "Formatted at 2026-08-12 03:18:05 on dist-test-slave-7nm7"
server_key: "d1b2c165106ab9e532792aea79b8b62c"
server_key_iv: "17c43f7e1ea21b3db43a853d176b74f5"
server_key_version: "encryptionkey@0"
I20260812 03:18:05.273728 24524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:18:05.278721 24524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:18:05.281983 24524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:18:05.282205 24524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:18:05.289388 24524 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:43095
I20260812 03:18:05.289448 24640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:43095 every 8 connection(s)
I20260812 03:18:05.290452 24524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:18:05.293411 24643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:18:05.298596 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 24524
I20260812 03:18:05.298717 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:18:05.298981 17253 external_mini_cluster.cc:1428] Setting key fb98eb4f3a4093cf185300c053929c06
I20260812 03:18:05.299582 24643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63: Bootstrap starting.
I20260812 03:18:05.302304 24643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63: Neither blocks nor log segments found. Creating new log.
I20260812 03:18:05.303179 24643 log.cc:824] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63: Log is configured to *not* fsync() on all Append() calls
I20260812 03:18:05.305136 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:05.300456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34879 (local address 127.16.217.126:43095)
0812 03:18:05.301364 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:05.301391 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:05.301447 (+ 56us) server_negotiation.cc:407] Connection header received
0812 03:18:05.302160 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:05.302180 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:05.302597 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:05.302949 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:05.303793 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:05.303800 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:05.303819 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:18:05.303830 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:05.303889 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:18:05.303910 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:05.304032 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:05.304143 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 03:18:05.304386 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":151,"threads_started":1}
I20260812 03:18:05.305498 24643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63: No bootstrap required, opened a new log
I20260812 03:18:05.308467 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:18:05.310029 24643 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec9a526c492b42659c97328aa9999b63" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43095 } }
I20260812 03:18:05.310333 24643 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:18:05.310422 24643 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec9a526c492b42659c97328aa9999b63, State: Initialized, Role: FOLLOWER
I20260812 03:18:05.310992 24643 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec9a526c492b42659c97328aa9999b63" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43095 } }
I20260812 03:18:05.311188 24643 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:18:05.311266 24643 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:18:05.311388 24643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:18:05.312628 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:05.310621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39677 (local address 127.16.217.126:43095)
0812 03:18:05.310768 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:05.310773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:05.310986 (+ 213us) server_negotiation.cc:407] Connection header received
0812 03:18:05.311065 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:05.311069 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:05.311112 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:05.311219 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:05.312152 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:05.312156 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:05.312159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:18:05.312173 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:05.312205 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:18:05.312222 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:05.312327 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:05.312440 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 03:18:05.312486 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:18:05.312819 24643 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec9a526c492b42659c97328aa9999b63" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43095 } }
I20260812 03:18:05.313225 24643 leader_election.cc:302] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec9a526c492b42659c97328aa9999b63; no voters:
I20260812 03:18:05.313612 24643 leader_election.cc:288] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:18:05.313723 24661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:18:05.313974 24661 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63 [term 1 LEADER]: Becoming Leader. State: Replica: ec9a526c492b42659c97328aa9999b63, State: Running, Role: LEADER
I20260812 03:18:05.314462 24661 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec9a526c492b42659c97328aa9999b63" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43095 } }
I20260812 03:18:05.315058 24643 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:18:05.317523 24664 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec9a526c492b42659c97328aa9999b63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec9a526c492b42659c97328aa9999b63" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43095 } } }
I20260812 03:18:05.317688 24664 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63 [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:05.318045 24665 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec9a526c492b42659c97328aa9999b63. Latest consensus state: current_term: 1 leader_uuid: "ec9a526c492b42659c97328aa9999b63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec9a526c492b42659c97328aa9999b63" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43095 } } }
I20260812 03:18:05.318512 24665 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63 [sys.catalog]: This master's current role is: LEADER
W20260812 03:18:05.320719 24689 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:18:05.320796 24689 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:18:05.320884 24690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:18:05.321739 24690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:18:05.329241 24690 catalog_manager.cc:1347] Generated new cluster ID: dc0dce06fe8e40c38364c3d460cffb74
I20260812 03:18:05.329330 24690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:18:05.338125 24690 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:18:05.338544 24690 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:18:05.351841 24690 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec9a526c492b42659c97328aa9999b63: Generated new TSK 0
I20260812 03:18:05.352782 24690 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:18:05.359935 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 24524
2026-08-12T03:18:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1751 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:18:05 dist-test-slave-7nm7 krb5kdc[24703](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:18:05 dist-test-slave-7nm7 krb5kdc[24703](info): set up 2 sockets
Aug 12 03:18:05 dist-test-slave-7nm7 krb5kdc[24703](info): commencing operation
krb5kdc: starting...
W20260812 03:18:06.707676 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:18:06 dist-test-slave-7nm7 krb5kdc[24703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:18:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:18:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:18:06.841081 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:34477
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:36403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:34477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:18:06.957829 24722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:18:06.958117 24722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:18:06.958163 24722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:18:06.962204 24722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:18:06.962303 24722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:18:06.962327 24722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:18:06.962347 24722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:18:06.962365 24722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:18:06.967769 24722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:36403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:34477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:34477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:18:06.968884 24722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:18:06.969914 24722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:18:06.978821 24732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:18:06.978850 24731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:18:06.978859 24735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:18:06.979097 24722 server_base.cc:1034] running on GCE node
I20260812 03:18:06.979710 24722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:18:06.980901 24722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:18:06.982108 24722 hybrid_clock.cc:648] HybridClock initialized: now 1786504686982088 us; error 33 us; skew 500 ppm
Aug 12 03:18:06 dist-test-slave-7nm7 krb5kdc[24703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504686, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:18:06.984956 24722 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:18:06.985843 24722 webserver.cc:460] Webserver started at http://127.16.217.126:36847/ using document root <none> and password file <none>
I20260812 03:18:06.986454 24722 fs_manager.cc:359] Metadata directory not provided
I20260812 03:18:06.986521 24722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:18:06.986693 24722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:18:06.988782 24722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "36ae720941714b429efc98aa00c58e26"
format_stamp: "Formatted at 2026-08-12 03:18:06 on dist-test-slave-7nm7"
server_key: "7a08354a0f059070b0f45e97ac9385a1"
server_key_iv: "062adb462efda51b8c35eedaa43b371a"
server_key_version: "encryptionkey@0"
I20260812 03:18:06.989319 24722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "36ae720941714b429efc98aa00c58e26"
format_stamp: "Formatted at 2026-08-12 03:18:06 on dist-test-slave-7nm7"
server_key: "7a08354a0f059070b0f45e97ac9385a1"
server_key_iv: "062adb462efda51b8c35eedaa43b371a"
server_key_version: "encryptionkey@0"
I20260812 03:18:06.993124 24722 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 03:18:06.995843 24751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:18:06.997578 24722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:18:06.997701 24722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "36ae720941714b429efc98aa00c58e26"
format_stamp: "Formatted at 2026-08-12 03:18:06 on dist-test-slave-7nm7"
server_key: "7a08354a0f059070b0f45e97ac9385a1"
server_key_iv: "062adb462efda51b8c35eedaa43b371a"
server_key_version: "encryptionkey@0"
I20260812 03:18:06.997798 24722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:18:07.015349 24722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:18:07.019375 24722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:18:07.019562 24722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:18:07.027841 24722 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:34477
I20260812 03:18:07.027913 24836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:34477 every 8 connection(s)
I20260812 03:18:07.028875 24722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:18:07.031927 24838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:18:07.038358 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 24722
I20260812 03:18:07.038503 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:18:07.038700 24838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26: Bootstrap starting.
I20260812 03:18:07.038828 17253 external_mini_cluster.cc:1428] Setting key 50221f60252fba5a9ade74bd86b9af8b
I20260812 03:18:07.041812 24838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26: Neither blocks nor log segments found. Creating new log.
I20260812 03:18:07.042896 24838 log.cc:824] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26: Log is configured to *not* fsync() on all Append() calls
I20260812 03:18:07.045435 24838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26: No bootstrap required, opened a new log
Aug 12 03:18:07 dist-test-slave-7nm7 krb5kdc[24703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:18:07.049748 24838 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36ae720941714b429efc98aa00c58e26" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 34477 } }
I20260812 03:18:07.050096 24838 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:18:07.050168 24838 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36ae720941714b429efc98aa00c58e26, State: Initialized, Role: FOLLOWER
I20260812 03:18:07.050758 24838 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36ae720941714b429efc98aa00c58e26" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 34477 } }
I20260812 03:18:07.050976 24838 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:18:07.051050 24838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:18:07.051177 24838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:18:07.052534 24838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36ae720941714b429efc98aa00c58e26" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 34477 } }
I20260812 03:18:07.052927 24838 leader_election.cc:302] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36ae720941714b429efc98aa00c58e26; no voters:
I20260812 03:18:07.053675 24838 leader_election.cc:288] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:18:07.053767 24845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:18:07.054021 24845 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26 [term 1 LEADER]: Becoming Leader. State: Replica: 36ae720941714b429efc98aa00c58e26, State: Running, Role: LEADER
I20260812 03:18:07.054386 24845 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36ae720941714b429efc98aa00c58e26" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 34477 } }
I20260812 03:18:07.055217 24838 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:18:07.055728 24841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:07.040248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45439 (local address 127.16.217.126:34477)
0812 03:18:07.040717 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:07.040737 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:07.040779 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:18:07.041953 (+ 1174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:07.041980 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:07.042493 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:07.042890 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:07.043906 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:07.045432 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:07.046402 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:07.047092 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:07.047263 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:07.049884 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:07.049891 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:07.049909 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:07.049942 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:07.052186 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:07.052745 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:07.052751 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:07.052757 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:07.052903 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:07.053273 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:07.053276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:07.053278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:07.053715 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:07.054046 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:07.054414 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 03:18:07.054751 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":135,"threads_started":1}
I20260812 03:18:07.055809 24848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36ae720941714b429efc98aa00c58e26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36ae720941714b429efc98aa00c58e26" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 34477 } } }
I20260812 03:18:07.055832 24849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36ae720941714b429efc98aa00c58e26. Latest consensus state: current_term: 1 leader_uuid: "36ae720941714b429efc98aa00c58e26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36ae720941714b429efc98aa00c58e26" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 34477 } } }
I20260812 03:18:07.055934 24849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26 [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:07.055933 24848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26 [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:07.057916 24853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:18:07.059029 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:18:07.061255 24853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:18:07.068292 24853 catalog_manager.cc:1347] Generated new cluster ID: 9952825206d24b7da2ded3403eb37273
I20260812 03:18:07.068420 24853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:18:07.070722 24841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:07.061264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34227 (local address 127.16.217.126:34477)
0812 03:18:07.061435 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:07.061439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:07.061945 (+ 506us) server_negotiation.cc:407] Connection header received
0812 03:18:07.062133 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:07.062136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:07.062185 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:07.062273 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:07.063244 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:07.063838 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:07.064637 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:07.064930 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:07.065062 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:07.067069 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:07.067073 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:07.067075 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:07.067101 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:07.068869 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:07.069401 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:07.069404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:07.069406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:07.069490 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:07.069805 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:07.069808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:07.069810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:07.070002 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:07.070115 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:07.070447 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 03:18:07.070566 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 03:18:07.089113 24853 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:18:07.090193 24853 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:18:07.100274 24853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36ae720941714b429efc98aa00c58e26: Generated new TSK 0
I20260812 03:18:07.101102 24853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:18:07.109017 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 24722
2026-08-12T03:18:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1751 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:18:07 dist-test-slave-7nm7 krb5kdc[24877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:18:07 dist-test-slave-7nm7 krb5kdc[24877](info): set up 2 sockets
Aug 12 03:18:07 dist-test-slave-7nm7 krb5kdc[24877](info): commencing operation
krb5kdc: starting...
W20260812 03:18:08.464046 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.314s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:18:08 dist-test-slave-7nm7 krb5kdc[24877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:18:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:18:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:18:08.608038 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:43179
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:38337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:43179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:18:08.723348 24907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:18:08.723619 24907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:18:08.723661 24907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:18:08.727756 24907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:18:08.727823 24907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:18:08.727842 24907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:18:08.727859 24907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:18:08.727912 24907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:18:08.733239 24907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:38337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:43179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:43179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:18:08.734481 24907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:18:08.735579 24907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:18:08.745537 24920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:18:08.745607 24924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:18:08.745517 24931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:18:08.745998 24907 server_base.cc:1034] running on GCE node
I20260812 03:18:08.746695 24907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:18:08.747805 24907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:18:08.748970 24907 hybrid_clock.cc:648] HybridClock initialized: now 1786504688748959 us; error 44 us; skew 500 ppm
Aug 12 03:18:08 dist-test-slave-7nm7 krb5kdc[24877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504688, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:18:08.757094 24907 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:18:08.758164 24907 webserver.cc:460] Webserver started at http://127.16.217.126:46739/ using document root <none> and password file <none>
I20260812 03:18:08.758811 24907 fs_manager.cc:359] Metadata directory not provided
I20260812 03:18:08.758909 24907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:18:08.759158 24907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:18:08.761300 24907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "ceef4e41d674400db0215cd4314d0f2c"
format_stamp: "Formatted at 2026-08-12 03:18:08 on dist-test-slave-7nm7"
server_key: "eb210058aa1d5660e0aa4fb6c39f2332"
server_key_iv: "e950dd0d928979f353ee81a180e583e5"
server_key_version: "encryptionkey@0"
I20260812 03:18:08.761893 24907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "ceef4e41d674400db0215cd4314d0f2c"
format_stamp: "Formatted at 2026-08-12 03:18:08 on dist-test-slave-7nm7"
server_key: "eb210058aa1d5660e0aa4fb6c39f2332"
server_key_iv: "e950dd0d928979f353ee81a180e583e5"
server_key_version: "encryptionkey@0"
I20260812 03:18:08.765846 24907 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:18:08.768541 24942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:18:08.769837 24907 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:18:08.769989 24907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "ceef4e41d674400db0215cd4314d0f2c"
format_stamp: "Formatted at 2026-08-12 03:18:08 on dist-test-slave-7nm7"
server_key: "eb210058aa1d5660e0aa4fb6c39f2332"
server_key_iv: "e950dd0d928979f353ee81a180e583e5"
server_key_version: "encryptionkey@0"
I20260812 03:18:08.770112 24907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:18:08.805660 24907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:18:08.808951 24907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:18:08.809190 24907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:18:08.817090 24907 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:43179
I20260812 03:18:08.817147 25039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:43179 every 8 connection(s)
I20260812 03:18:08.818149 24907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:18:08.821107 25040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:18:08.826713 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 24907
I20260812 03:18:08.826871 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:18:08.827157 17253 external_mini_cluster.cc:1428] Setting key c10b2a7280377c4aca80659ce9b50918
I20260812 03:18:08.827844 25040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c: Bootstrap starting.
I20260812 03:18:08.830423 25040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c: Neither blocks nor log segments found. Creating new log.
I20260812 03:18:08.831223 25040 log.cc:824] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:18:08.833153 25040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c: No bootstrap required, opened a new log
I20260812 03:18:08.836124 25044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:08.828577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59621 (local address 127.16.217.126:43179)
0812 03:18:08.829053 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:08.829068 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:08.829104 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:18:08.829743 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:08.829759 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:08.830092 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:08.830461 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:08.830896 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:08.831908 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:08.832811 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:08.833331 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:08.833501 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:08.834450 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:08.834455 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:08.834468 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:18:08.834866 (+ 398us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:08.834921 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:18:08.834944 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:08.835064 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:08.835195 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 03:18:08.835405 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":150,"threads_started":1}
I20260812 03:18:08.837170 25040 raft_consensus.cc:348] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceef4e41d674400db0215cd4314d0f2c" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43179 } }
I20260812 03:18:08.837459 25040 raft_consensus.cc:374] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:18:08.837523 25040 raft_consensus.cc:729] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceef4e41d674400db0215cd4314d0f2c, State: Initialized, Role: FOLLOWER
I20260812 03:18:08.837980 25040 consensus_queue.cc:260] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceef4e41d674400db0215cd4314d0f2c" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43179 } }
I20260812 03:18:08.838191 25040 raft_consensus.cc:388] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:18:08.838292 25040 raft_consensus.cc:482] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:18:08.838403 25040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:18:08.839171 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:18:08.839553 25040 raft_consensus.cc:504] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceef4e41d674400db0215cd4314d0f2c" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43179 } }
I20260812 03:18:08.839896 25040 leader_election.cc:302] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ceef4e41d674400db0215cd4314d0f2c; no voters:
I20260812 03:18:08.840250 25040 leader_election.cc:288] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:18:08.840322 25046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:18:08.840550 25046 raft_consensus.cc:686] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c [term 1 LEADER]: Becoming Leader. State: Replica: ceef4e41d674400db0215cd4314d0f2c, State: Running, Role: LEADER
I20260812 03:18:08.840876 25046 consensus_queue.cc:237] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceef4e41d674400db0215cd4314d0f2c" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43179 } }
I20260812 03:18:08.841605 25040 sys_catalog.cc:564] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:18:08.842608 25054 sys_catalog.cc:455] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ceef4e41d674400db0215cd4314d0f2c. Latest consensus state: current_term: 1 leader_uuid: "ceef4e41d674400db0215cd4314d0f2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceef4e41d674400db0215cd4314d0f2c" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43179 } } }
I20260812 03:18:08.842749 25054 sys_catalog.cc:458] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:08.843076 25050 sys_catalog.cc:455] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ceef4e41d674400db0215cd4314d0f2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceef4e41d674400db0215cd4314d0f2c" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 43179 } } }
I20260812 03:18:08.843158 25050 sys_catalog.cc:458] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:08.843144 25071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:18:08.847668 25044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:08.841393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34873 (local address 127.16.217.126:43179)
0812 03:18:08.841501 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:08.841504 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:08.841926 (+ 422us) server_negotiation.cc:407] Connection header received
0812 03:18:08.843379 (+ 1453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:08.843382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:08.843437 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:08.843494 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:08.843960 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:08.844779 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:08.845937 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:08.846166 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:08.846313 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:08.847166 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:08.847170 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:08.847172 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:18:08.847185 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:08.847211 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:18:08.847221 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:08.847307 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:08.847407 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 03:18:08.847458 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 03:18:08.847520 25071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:18:08.853892 25071 catalog_manager.cc:1347] Generated new cluster ID: 1f64ac640b8c4da9b8beeef079433e09
I20260812 03:18:08.853972 25071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:18:08.868669 25071 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:18:08.869648 25071 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:18:08.874894 25071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ceef4e41d674400db0215cd4314d0f2c: Generated new TSK 0
I20260812 03:18:08.875728 25071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:18:08.882936 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 24907
2026-08-12T03:18:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1769 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:18:08 dist-test-slave-7nm7 krb5kdc[25090](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:18:08 dist-test-slave-7nm7 krb5kdc[25090](info): set up 2 sockets
Aug 12 03:18:08 dist-test-slave-7nm7 krb5kdc[25090](info): commencing operation
krb5kdc: starting...
W20260812 03:18:10.205978 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:18:10 dist-test-slave-7nm7 krb5kdc[25090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:18:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:18:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:18:10.337579 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:44013
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:37457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:44013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:18:10.459939 25112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:18:10.460260 25112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:18:10.460345 25112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:18:10.464463 25112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:18:10.464552 25112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:18:10.464591 25112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:18:10.464651 25112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:18:10.464704 25112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:18:10.470165 25112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:37457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:44013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:44013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:18:10.471360 25112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:18:10.472420 25112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:18:10.481339 25112 server_base.cc:1034] running on GCE node
W20260812 03:18:10.481261 25122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:18:10.481261 25123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:18:10.481279 25125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:18:10.482271 25112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:18:10.483560 25112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:18:10.484746 25112 hybrid_clock.cc:648] HybridClock initialized: now 1786504690484697 us; error 59 us; skew 500 ppm
Aug 12 03:18:10 dist-test-slave-7nm7 krb5kdc[25090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504690, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:18:10.487771 25112 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:18:10.488724 25112 webserver.cc:460] Webserver started at http://127.16.217.126:42897/ using document root <none> and password file <none>
I20260812 03:18:10.489301 25112 fs_manager.cc:359] Metadata directory not provided
I20260812 03:18:10.489365 25112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:18:10.489598 25112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:18:10.491761 25112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "7ceb654cc63f4d73889aeb7519409e77"
format_stamp: "Formatted at 2026-08-12 03:18:10 on dist-test-slave-7nm7"
server_key: "587fc332e9f80929841b2c3b04cc5c6f"
server_key_iv: "c3f6f5528f51565cf5edb9d177aae137"
server_key_version: "encryptionkey@0"
I20260812 03:18:10.492345 25112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "7ceb654cc63f4d73889aeb7519409e77"
format_stamp: "Formatted at 2026-08-12 03:18:10 on dist-test-slave-7nm7"
server_key: "587fc332e9f80929841b2c3b04cc5c6f"
server_key_iv: "c3f6f5528f51565cf5edb9d177aae137"
server_key_version: "encryptionkey@0"
I20260812 03:18:10.496266 25112 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:18:10.498826 25138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:18:10.500320 25112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:18:10.500484 25112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "7ceb654cc63f4d73889aeb7519409e77"
format_stamp: "Formatted at 2026-08-12 03:18:10 on dist-test-slave-7nm7"
server_key: "587fc332e9f80929841b2c3b04cc5c6f"
server_key_iv: "c3f6f5528f51565cf5edb9d177aae137"
server_key_version: "encryptionkey@0"
I20260812 03:18:10.500612 25112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:18:10.505460 25112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:18:10.508522 25112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:18:10.508744 25112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:18:10.516731 25112 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:44013
I20260812 03:18:10.516760 25238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:44013 every 8 connection(s)
I20260812 03:18:10.517844 25112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:18:10.520768 25243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:18:10.525813 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 25112
I20260812 03:18:10.525964 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:18:10.526306 17253 external_mini_cluster.cc:1428] Setting key 7255e918c3d22303ae3106112ee67645
I20260812 03:18:10.526665 25243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77: Bootstrap starting.
I20260812 03:18:10.529279 25243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77: Neither blocks nor log segments found. Creating new log.
I20260812 03:18:10.530179 25243 log.cc:824] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77: Log is configured to *not* fsync() on all Append() calls
I20260812 03:18:10.531715 25247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:10.527748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35075 (local address 127.16.217.126:44013)
0812 03:18:10.528226 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:10.528243 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:10.528276 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:18:10.528906 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:10.528922 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:10.529205 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:10.529553 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:10.530393 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:10.530398 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:10.530410 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:18:10.530421 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:10.530474 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:18:10.530493 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:10.530609 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:10.530722 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 03:18:10.530883 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":158,"threads_started":1}
I20260812 03:18:10.532272 25243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77: No bootstrap required, opened a new log
I20260812 03:18:10.534567 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:18:10.536780 25243 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ceb654cc63f4d73889aeb7519409e77" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44013 } }
I20260812 03:18:10.537081 25243 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:18:10.537156 25243 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ceb654cc63f4d73889aeb7519409e77, State: Initialized, Role: FOLLOWER
I20260812 03:18:10.537640 25243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ceb654cc63f4d73889aeb7519409e77" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44013 } }
I20260812 03:18:10.537863 25243 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:18:10.537946 25243 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:18:10.538053 25243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:18:10.539108 25247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:10.537319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45347 (local address 127.16.217.126:44013)
0812 03:18:10.537465 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:10.537471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:10.537639 (+ 168us) server_negotiation.cc:407] Connection header received
0812 03:18:10.537768 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:10.537770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:10.537826 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:10.537932 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:10.538714 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:10.538717 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:10.538720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:18:10.538731 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:10.538759 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:18:10.538775 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:10.538865 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:10.538939 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 03:18:10.538981 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:18:10.539250 25243 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ceb654cc63f4d73889aeb7519409e77" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44013 } }
I20260812 03:18:10.539630 25243 leader_election.cc:302] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ceb654cc63f4d73889aeb7519409e77; no voters:
I20260812 03:18:10.540017 25243 leader_election.cc:288] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:18:10.540103 25258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:18:10.540354 25258 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77 [term 1 LEADER]: Becoming Leader. State: Replica: 7ceb654cc63f4d73889aeb7519409e77, State: Running, Role: LEADER
I20260812 03:18:10.540717 25258 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ceb654cc63f4d73889aeb7519409e77" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44013 } }
I20260812 03:18:10.541301 25243 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:18:10.542555 25261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ceb654cc63f4d73889aeb7519409e77. Latest consensus state: current_term: 1 leader_uuid: "7ceb654cc63f4d73889aeb7519409e77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ceb654cc63f4d73889aeb7519409e77" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44013 } } }
I20260812 03:18:10.542683 25261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77 [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:10.543083 25275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:18:10.543074 25259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ceb654cc63f4d73889aeb7519409e77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ceb654cc63f4d73889aeb7519409e77" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 44013 } } }
I20260812 03:18:10.543890 25259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77 [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:10.546681 25275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:18:10.551954 25275 catalog_manager.cc:1347] Generated new cluster ID: a7ddd7f133f84c6ebeb41dee1654363c
I20260812 03:18:10.552022 25275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:18:10.573825 25275 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:18:10.574173 25275 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:18:10.578907 25275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ceb654cc63f4d73889aeb7519409e77: Generated new TSK 0
I20260812 03:18:10.579712 25275 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:18:10.584180 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 25112
2026-08-12T03:18:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1702 ms)
[----------] 18 tests from AuthTokenIssuingTest (31517 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:18:10 dist-test-slave-7nm7 krb5kdc[25293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:18:10 dist-test-slave-7nm7 krb5kdc[25293](info): set up 2 sockets
Aug 12 03:18:10 dist-test-slave-7nm7 krb5kdc[25293](info): commencing operation
krb5kdc: starting...
W20260812 03:18:11.902710 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:18:11 dist-test-slave-7nm7 krb5kdc[25293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:18:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:18:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:18:12.038173 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:36203
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:36203,127.16.217.125:46557,127.16.217.124:43355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:18:12.154803 25316 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:18:12.155170 25316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:18:12.155249 25316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:18:12.155333 25316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:18:12.159425 25316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:18:12.159528 25316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:18:12.159566 25316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:18:12.159605 25316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:18:12.159639 25316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:18:12.165174 25316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:36203,127.16.217.125:46557,127.16.217.124:43355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:36203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:18:12.166441 25316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:18:12.167541 25316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:18:12.176671 25323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:18:12.176612 25330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:18:12.176674 25326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:18:12.176930 25316 server_base.cc:1034] running on GCE node
I20260812 03:18:12.177734 25316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:18:12.179351 25316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:18:12.180760 25316 hybrid_clock.cc:648] HybridClock initialized: now 1786504692180746 us; error 50 us; skew 500 ppm
Aug 12 03:18:12 dist-test-slave-7nm7 krb5kdc[25293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504692, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:18:12.184230 25316 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:18:12.185310 25316 webserver.cc:460] Webserver started at http://127.16.217.126:46127/ using document root <none> and password file <none>
I20260812 03:18:12.185927 25316 fs_manager.cc:359] Metadata directory not provided
I20260812 03:18:12.186038 25316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:18:12.186347 25316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:18:12.188498 25316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "43ee8da813f3489d9fd360126acad442"
format_stamp: "Formatted at 2026-08-12 03:18:12 on dist-test-slave-7nm7"
server_key: "030d5900ba2e3ff8155cda5a93ec731e"
server_key_iv: "b9dd9dec0372230ac77117a25b348de1"
server_key_version: "encryptionkey@0"
I20260812 03:18:12.189106 25316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "43ee8da813f3489d9fd360126acad442"
format_stamp: "Formatted at 2026-08-12 03:18:12 on dist-test-slave-7nm7"
server_key: "030d5900ba2e3ff8155cda5a93ec731e"
server_key_iv: "b9dd9dec0372230ac77117a25b348de1"
server_key_version: "encryptionkey@0"
I20260812 03:18:12.193135 25316 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:18:12.195971 25346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:18:12.198514 25316 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:18:12.198704 25316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "43ee8da813f3489d9fd360126acad442"
format_stamp: "Formatted at 2026-08-12 03:18:12 on dist-test-slave-7nm7"
server_key: "030d5900ba2e3ff8155cda5a93ec731e"
server_key_iv: "b9dd9dec0372230ac77117a25b348de1"
server_key_version: "encryptionkey@0"
I20260812 03:18:12.198848 25316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:18:12.228245 25316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:18:12.231542 25316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:18:12.231775 25316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:18:12.239207 25316 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:36203
I20260812 03:18:12.239528 25440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:36203 every 8 connection(s)
I20260812 03:18:12.240260 25316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:18:12.243847 25441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:18:12.246848 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 25316
I20260812 03:18:12.246994 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:18:12.247272 17253 external_mini_cluster.cc:1428] Setting key 2927732a900415d23f76f070b9c65934
I20260812 03:18:12.248616 25441 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.16.217.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.125@KRBTEST.COM" created.
Aug 12 03:18:12 dist-test-slave-7nm7 krb5kdc[25293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504692, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:18:12.265000 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.251294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:36203 (local address 127.0.0.1:38696)
0812 03:18:12.251799 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.251853 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:18:12.252782 (+ 929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:18:12.253458 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:18:12.253463 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:18:12.253914 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:18:12.254150 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:12.254163 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.255401 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.255405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:12.256098 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:12.256108 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.256794 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.256798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:12.256893 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.257494 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:18:12.257511 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:18:12.259730 (+ 2219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:18:12.261700 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:12.261711 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:12.261721 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:12.261949 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:12.262347 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:12.262350 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:12.262352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:12.262470 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:12.263289 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:18:12.263295 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:18:12.263439 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 03:18:12.263618 (+ 179us) client_negotiation.cc:240] Negotiation successful
0812 03:18:12.264017 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":108,"thread_start_us":150,"threads_started":1}
I20260812 03:18:12.265000 25447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.251295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38696 (local address 127.16.217.126:36203)
0812 03:18:12.252298 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.252310 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.252352 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:18:12.253090 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.253095 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.253272 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.253458 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.254374 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.255252 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.256257 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.256626 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.256808 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.259895 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:12.259901 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:12.259911 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:12.259933 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:12.261575 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.262067 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.262075 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.262081 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.262189 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.262651 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.262655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.262657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.263159 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:12.263308 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:12.263682 (+ 374us) server_negotiation.cc:299] Negotiation successful
0812 03:18:12.263936 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":889,"spinlock_wait_cycles":17280,"thread_start_us":151,"threads_started":1}
I20260812 03:18:12.267371 25441 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:18:12.268177 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.267872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:46557 (local address 127.0.0.1:51284)
0812 03:18:12.268041 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.268111 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:46557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260812 03:18:12.268392 25348 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.217.125:46557: connect: Connection refused (error 111)
I20260812 03:18:12.269289 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.269041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:46557 (local address 127.0.0.1:51298)
0812 03:18:12.269136 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.269216 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:46557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 03:18:12.269464 25441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.125:46557: Network error: Client connection negotiation failed: client connection to 127.16.217.125:46557: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.125.keytab.
Entry for principal kudu/127.16.217.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.125.keytab.
I20260812 03:18:12.274467 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.125:46557
--webserver_interface=127.16.217.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.125.keytab
--principal=kudu/127.16.217.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:36203,127.16.217.125:46557,127.16.217.124:43355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:18:12.318759 25441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } attempt: 1
I20260812 03:18:12.319882 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.319514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:46557 (local address 127.0.0.1:51306)
0812 03:18:12.319705 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.319776 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:46557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 03:18:12.320822 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.320568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:46557 (local address 127.0.0.1:51320)
0812 03:18:12.320698 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.320754 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:46557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 03:18:12.321007 25441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.125:46557: Network error: Client connection negotiation failed: client connection to 127.16.217.125:46557: connect: Connection refused (error 111)
I20260812 03:18:12.389390 25441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } attempt: 2
I20260812 03:18:12.390686 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.390320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:46557 (local address 127.0.0.1:51328)
0812 03:18:12.390502 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.390573 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:46557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 03:18:12.391602 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.391335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:46557 (local address 127.0.0.1:51332)
0812 03:18:12.391479 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.391535 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:46557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 03:18:12.391777 25441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.125:46557: Network error: Client connection negotiation failed: client connection to 127.16.217.125:46557: connect: Connection refused (error 111)
W20260812 03:18:12.395707 25450 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:18:12.396088 25450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:18:12.396144 25450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:18:12.396215 25450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:18:12.400338 25450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:18:12.400393 25450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:18:12.400413 25450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:18:12.400430 25450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:18:12.400485 25450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:18:12.406100 25450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:36203,127.16.217.125:46557,127.16.217.124:43355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.125.keytab
--principal=kudu/127.16.217.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.125:46557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.217.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:18:12.407231 25450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:18:12.408248 25450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:18:12.417913 25461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:18:12.417959 25459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:18:12.418298 25450 server_base.cc:1034] running on GCE node
W20260812 03:18:12.419658 25464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:18:12.420609 25450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:18:12.421970 25450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:18:12.424959 25450 hybrid_clock.cc:648] HybridClock initialized: now 1786504692424906 us; error 70 us; skew 500 ppm
Aug 12 03:18:12 dist-test-slave-7nm7 krb5kdc[25293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504692, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:18:12.428880 25450 init.cc:377] Logged in from keytab as kudu/127.16.217.125@KRBTEST.COM (short username kudu)
I20260812 03:18:12.429849 25450 webserver.cc:460] Webserver started at http://127.16.217.125:33755/ using document root <none> and password file <none>
I20260812 03:18:12.430526 25450 fs_manager.cc:359] Metadata directory not provided
I20260812 03:18:12.430630 25450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:18:12.430868 25450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:18:12.433107 25450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-1/data/instance:
uuid: "7bf4ef56b59f4589a3e8831077b6728e"
format_stamp: "Formatted at 2026-08-12 03:18:12 on dist-test-slave-7nm7"
server_key: "9ead5d92b8c7a833fb9a1636a8d0223d"
server_key_iv: "2e1fc2cec2da9fd218879f00d61be1e1"
server_key_version: "encryptionkey@0"
I20260812 03:18:12.433725 25450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-1/wal/instance:
uuid: "7bf4ef56b59f4589a3e8831077b6728e"
format_stamp: "Formatted at 2026-08-12 03:18:12 on dist-test-slave-7nm7"
server_key: "9ead5d92b8c7a833fb9a1636a8d0223d"
server_key_iv: "2e1fc2cec2da9fd218879f00d61be1e1"
server_key_version: "encryptionkey@0"
I20260812 03:18:12.437879 25450 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 03:18:12.441588 25473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:18:12.443390 25450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:18:12.443533 25450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-1/wal
uuid: "7bf4ef56b59f4589a3e8831077b6728e"
format_stamp: "Formatted at 2026-08-12 03:18:12 on dist-test-slave-7nm7"
server_key: "9ead5d92b8c7a833fb9a1636a8d0223d"
server_key_iv: "2e1fc2cec2da9fd218879f00d61be1e1"
server_key_version: "encryptionkey@0"
I20260812 03:18:12.443653 25450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:18:12.463126 25450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:18:12.466466 25450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:18:12.466638 25450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:18:12.474517 25450 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.125:46557
I20260812 03:18:12.474828 25558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.125:46557 every 8 connection(s)
I20260812 03:18:12.475540 25450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-1/data/info.pb
I20260812 03:18:12.478623 25560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:18:12.482105 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 25450
I20260812 03:18:12.482220 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-1/wal/instance
I20260812 03:18:12.482604 25560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:18:12.482911 17253 external_mini_cluster.cc:1428] Setting key b48777b892ed8219d1b03c1c82fa0817
I20260812 03:18:12.483004 25441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } attempt: 3
Aug 12 03:18:12 dist-test-slave-7nm7 krb5kdc[25293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504692, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for kudu/127.16.217.125@KRBTEST.COM
Aug 12 03:18:12 dist-test-slave-7nm7 krb5kdc[25293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504692, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.125@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.217.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.124@KRBTEST.COM" created.
I20260812 03:18:12.497575 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.483550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:46557 (local address 127.0.0.1:51342)
0812 03:18:12.483757 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.483774 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:18:12.483869 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:18:12.486512 (+ 2643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:18:12.486514 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:18:12.486713 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:18:12.486804 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:12.486810 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.488242 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.488248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:12.489258 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:12.489274 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.490013 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.490017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:12.490125 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.490718 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:18:12.490731 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:18:12.492379 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:18:12.495099 (+ 2720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:12.495102 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:12.495106 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:12.495548 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:12.496024 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:12.496028 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:12.496031 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:12.496162 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:12.496925 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:18:12.496928 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:18:12.497035 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 03:18:12.497135 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 03:18:12.497302 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260812 03:18:12.497915 25447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.484565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38702 (local address 127.16.217.126:36203)
0812 03:18:12.484706 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.484724 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.485289 (+ 565us) server_negotiation.cc:407] Connection header received
0812 03:18:12.486322 (+ 1033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.486326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.486382 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.486497 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.487890 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.488495 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.489628 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.490837 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.490976 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.493669 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:12.493674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:12.493678 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:12.493718 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:12.495635 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.496216 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.496223 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.496225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.496433 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.496820 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.496824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.496826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.497020 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:12.497251 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:12.497638 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 03:18:12.497763 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":45}
I20260812 03:18:12.499137 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.484364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51342 (local address 127.16.217.125:46557)
0812 03:18:12.484914 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.484933 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.484971 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:18:12.485999 (+ 1028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.486026 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.486329 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.486530 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.487001 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.488113 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.489400 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.489881 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.490057 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.492557 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:12.492566 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:12.492582 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:12.492614 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:12.494984 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.495744 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.495752 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.495756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.495911 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.496268 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.496273 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.496275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.496760 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:12.497974 (+ 1214us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:12.498087 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 03:18:12.498283 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":165,"threads_started":1}
I20260812 03:18:12.499171 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.484554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:36203 (local address 127.0.0.1:38702)
0812 03:18:12.485120 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.485178 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 03:18:12.486030 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:18:12.486955 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:18:12.486959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:18:12.487465 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:18:12.487722 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:12.487740 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.488623 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.488626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:12.489500 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:12.489509 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.491021 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.491025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:12.491146 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.491763 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:18:12.491784 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:18:12.493520 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:18:12.495726 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:12.495735 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:12.495746 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:12.495984 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:12.496569 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:12.496573 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:12.496577 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:12.496683 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:12.497253 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:18:12.497262 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:18:12.497403 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 03:18:12.497581 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 03:18:12.497916 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":137,"threads_started":1}
I20260812 03:18:12.500543 25560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:18:12.501520 25441 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:18:12.502321 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.502045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:43355 (local address 127.0.0.1:34726)
0812 03:18:12.502165 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.502220 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 03:18:12.503139 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.502744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:43355 (local address 127.0.0.1:34740)
0812 03:18:12.502987 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.503051 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
W20260812 03:18:12.503458 25441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.124:43355: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
Aug 12 03:18:12 dist-test-slave-7nm7 krb5kdc[25293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504692, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.125@KRBTEST.COM for kudu/127.16.217.125@KRBTEST.COM
I20260812 03:18:12.509181 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.501084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:46557 (local address 127.0.0.1:51358)
0812 03:18:12.501189 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.501203 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:18:12.501284 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:18:12.501497 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:18:12.501500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:18:12.501670 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:18:12.501761 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:12.501766 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.502761 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.502767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:12.503353 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:12.503361 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.503859 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.503862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:12.503949 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.504452 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:18:12.504466 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:18:12.505735 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:18:12.507414 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:12.507421 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:12.507423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:12.507749 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:12.508048 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:12.508050 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:12.508052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:12.508167 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:12.508597 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:18:12.508599 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:18:12.508688 (+ 89us) client_negotiation.cc:769] Sending connection context
0812 03:18:12.508833 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 03:18:12.508984 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 03:18:12.509187 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.501133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51358 (local address 127.16.217.125:46557)
0812 03:18:12.501248 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.501252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.501264 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:18:12.501356 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.501361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.501395 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.501488 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.501872 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.502640 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.503465 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.503748 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.503858 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.505861 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:12.505865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:12.505868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:12.505890 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:12.507304 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.507844 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.507847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.507849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.507928 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.508292 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.508296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.508298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.508488 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:12.508586 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:12.508880 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 03:18:12.509043 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":37}
I20260812 03:18:12.510077 25560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:18:12.510969 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.510623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:43355 (local address 127.0.0.1:34756)
0812 03:18:12.510762 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.510895 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 03:18:12.511695 25477 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.124.keytab.
Entry for principal kudu/127.16.217.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.124.keytab.
I20260812 03:18:12.512954 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.512697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:43355 (local address 127.0.0.1:34758)
0812 03:18:12.512828 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.512886 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 03:18:12.513211 25560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.124:43355: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
I20260812 03:18:12.513548 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.124:43355
--webserver_interface=127.16.217.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.124.keytab
--principal=kudu/127.16.217.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:36203,127.16.217.125:46557,127.16.217.124:43355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:18:12.534646 25441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } attempt: 1
I20260812 03:18:12.535647 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.535271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:43355 (local address 127.0.0.1:34774)
0812 03:18:12.535463 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.535540 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 03:18:12.536437 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.536196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:43355 (local address 127.0.0.1:34776)
0812 03:18:12.536313 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.536369 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 03:18:12.536585 25441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.124:43355: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
I20260812 03:18:12.562546 25560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } attempt: 1
I20260812 03:18:12.563586 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.563261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:43355 (local address 127.0.0.1:34790)
0812 03:18:12.563434 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.563496 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260812 03:18:12.564380 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.564160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:43355 (local address 127.0.0.1:34804)
0812 03:18:12.564270 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.564324 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 03:18:12.564543 25560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.124:43355: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
I20260812 03:18:12.611788 25441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } attempt: 2
I20260812 03:18:12.612790 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.612370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:43355 (local address 127.0.0.1:34814)
0812 03:18:12.612586 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.612673 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 03:18:12.613664 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.613444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:43355 (local address 127.0.0.1:34830)
0812 03:18:12.613564 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.613611 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 03:18:12.613898 25441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.124:43355: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
I20260812 03:18:12.632725 25560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } attempt: 2
I20260812 03:18:12.633606 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.633312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:43355 (local address 127.0.0.1:34838)
0812 03:18:12.633470 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.633530 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 03:18:12.634443 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.634189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:43355 (local address 127.0.0.1:34852)
0812 03:18:12.634345 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.634389 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260812 03:18:12.634583 25560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.124:43355: Network error: Client connection negotiation failed: client connection to 127.16.217.124:43355: connect: Connection refused (error 111)
W20260812 03:18:12.636610 25572 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:18:12.636886 25572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:18:12.636965 25572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:18:12.637068 25572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:18:12.641208 25572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:18:12.641263 25572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:18:12.641283 25572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:18:12.641299 25572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:18:12.641314 25572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:18:12.647111 25572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:36203,127.16.217.125:46557,127.16.217.124:43355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.124.keytab
--principal=kudu/127.16.217.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.124:43355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.217.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:18:12.648236 25572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:18:12.649365 25572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:18:12.658412 25587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:18:12.658481 25572 server_base.cc:1034] running on GCE node
W20260812 03:18:12.658420 25592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:18:12.658420 25583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:18:12.659240 25572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:18:12.660343 25572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:18:12.661537 25572 hybrid_clock.cc:648] HybridClock initialized: now 1786504692661501 us; error 51 us; skew 500 ppm
Aug 12 03:18:12 dist-test-slave-7nm7 krb5kdc[25293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504692, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:18:12.664517 25572 init.cc:377] Logged in from keytab as kudu/127.16.217.124@KRBTEST.COM (short username kudu)
I20260812 03:18:12.665508 25572 webserver.cc:460] Webserver started at http://127.16.217.124:36245/ using document root <none> and password file <none>
I20260812 03:18:12.666101 25572 fs_manager.cc:359] Metadata directory not provided
I20260812 03:18:12.666167 25572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:18:12.666472 25572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:18:12.668692 25572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-2/data/instance:
uuid: "17ab857126524c4ca09454c81a0643e6"
format_stamp: "Formatted at 2026-08-12 03:18:12 on dist-test-slave-7nm7"
server_key: "b3d41f817b13a6155076809bb5a34924"
server_key_iv: "6c9db1a80504f6b5329cdbf96bc5fbcb"
server_key_version: "encryptionkey@0"
I20260812 03:18:12.669364 25572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-2/wal/instance:
uuid: "17ab857126524c4ca09454c81a0643e6"
format_stamp: "Formatted at 2026-08-12 03:18:12 on dist-test-slave-7nm7"
server_key: "b3d41f817b13a6155076809bb5a34924"
server_key_iv: "6c9db1a80504f6b5329cdbf96bc5fbcb"
server_key_version: "encryptionkey@0"
I20260812 03:18:12.673480 25572 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260812 03:18:12.676147 25611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:18:12.677569 25572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:18:12.677691 25572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-2/wal
uuid: "17ab857126524c4ca09454c81a0643e6"
format_stamp: "Formatted at 2026-08-12 03:18:12 on dist-test-slave-7nm7"
server_key: "b3d41f817b13a6155076809bb5a34924"
server_key_iv: "6c9db1a80504f6b5329cdbf96bc5fbcb"
server_key_version: "encryptionkey@0"
I20260812 03:18:12.677790 25572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:18:12.698951 25572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:18:12.703830 25572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:18:12.704049 25572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:18:12.711659 25572 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.124:43355
I20260812 03:18:12.711964 25700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.124:43355 every 8 connection(s)
I20260812 03:18:12.712682 25572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-2/data/info.pb
I20260812 03:18:12.713068 25441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } attempt: 3
I20260812 03:18:12.716045 25701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:18:12.720422 25701 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:18:12 dist-test-slave-7nm7 krb5kdc[25293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504692, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for kudu/127.16.217.124@KRBTEST.COM
I20260812 03:18:12.722870 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 25572
I20260812 03:18:12.723021 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504382969880-17253-0/minicluster-data/master-2/wal/instance
I20260812 03:18:12.723397 17253 external_mini_cluster.cc:1428] Setting key 99fe35ab51398c3f7a5caab19f89630e
I20260812 03:18:12.725749 25560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } attempt: 3
I20260812 03:18:12.728183 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.713673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:43355 (local address 127.0.0.1:34858)
0812 03:18:12.713849 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.713872 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 03:18:12.713989 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:18:12.716806 (+ 2817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:18:12.716810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:18:12.717016 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:18:12.717138 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:12.717145 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.719073 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.719078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:12.719873 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:12.719887 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.720792 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.720796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:12.720878 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.721431 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:18:12.721447 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:18:12.723242 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:18:12.726024 (+ 2782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:12.726029 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:12.726033 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:12.726329 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:12.726764 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:12.726767 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:12.726769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:12.726864 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:12.727652 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:18:12.727656 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:18:12.727770 (+ 114us) client_negotiation.cc:769] Sending connection context
0812 03:18:12.727875 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 03:18:12.728008 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
Aug 12 03:18:12 dist-test-slave-7nm7 krb5kdc[25293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504692, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.124@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:18:12.730043 25711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.714591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34858 (local address 127.16.217.124:43355)
0812 03:18:12.715112 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.715133 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.715174 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:18:12.716153 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.716181 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.716520 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.716807 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.717497 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.718922 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.720018 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.720643 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.720851 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.723479 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:12.723487 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:12.723504 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:12.723541 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:12.725888 (+ 2347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.726470 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.726478 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.726485 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.726638 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.726967 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.726970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.726972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.727486 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:12.727603 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:12.728591 (+ 988us) server_negotiation.cc:299] Negotiation successful
0812 03:18:12.728906 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":152,"threads_started":1}
Aug 12 03:18:12 dist-test-slave-7nm7 krb5kdc[25293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.124@KRBTEST.COM
I20260812 03:18:12.735339 25447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.722684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38712 (local address 127.16.217.126:36203)
0812 03:18:12.723000 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.723006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.723469 (+ 463us) server_negotiation.cc:407] Connection header received
0812 03:18:12.723686 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.723691 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.723745 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.724174 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.725097 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.725612 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.726947 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.727291 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.727376 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.730954 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:12.730958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:12.730961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:12.730991 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:12.732857 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.733650 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.733654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.733656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.733746 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.734099 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.734102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.734104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.734445 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:12.734527 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:12.735018 (+ 491us) server_negotiation.cc:299] Negotiation successful
0812 03:18:12.735157 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201}
Aug 12 03:18:12 dist-test-slave-7nm7 krb5kdc[25293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504692, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.125@KRBTEST.COM for kudu/127.16.217.124@KRBTEST.COM
I20260812 03:18:12.736039 25713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.722513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:36203 (local address 127.0.0.1:38712)
0812 03:18:12.723330 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.723386 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 03:18:12.723565 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:18:12.724204 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:18:12.724210 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:18:12.724776 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:18:12.724975 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:12.724984 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.725733 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.725738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:12.726808 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:12.726818 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.727530 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.727533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:12.727667 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.728242 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:18:12.728259 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:18:12.730793 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:18:12.733008 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:12.733019 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:12.733028 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:12.733410 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:12.733874 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:12.733879 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:12.733883 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:12.733985 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:12.734615 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:18:12.734621 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:18:12.734765 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 03:18:12.735554 (+ 789us) client_negotiation.cc:240] Negotiation successful
0812 03:18:12.735837 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":154,"threads_started":1}
I20260812 03:18:12.736428 25716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.724447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34872 (local address 127.16.217.124:43355)
0812 03:18:12.724762 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.724769 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.724803 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:18:12.724971 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.724978 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.725029 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.725124 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.726524 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.727300 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.728451 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.728762 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.728869 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.731920 (+ 3051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:12.731928 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:12.731931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:12.731954 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:12.734020 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.734565 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.734568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.734570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.734656 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.734970 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.734973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.734975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.735187 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:12.735363 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:12.736125 (+ 762us) server_negotiation.cc:299] Negotiation successful
0812 03:18:12.736260 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":126,"threads_started":1}
I20260812 03:18:12.736524 25441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442: Bootstrap starting.
I20260812 03:18:12.737246 25701 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:18:12.740226 25441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442: Neither blocks nor log segments found. Creating new log.
I20260812 03:18:12.740562 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.726550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:43355 (local address 127.0.0.1:34886)
0812 03:18:12.726731 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.726744 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:18:12.726821 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:18:12.727539 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:18:12.727542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:18:12.727759 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:18:12.728167 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:12.728186 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.729995 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.730000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:12.730958 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:12.730969 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.733622 (+ 2653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.733629 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:12.733746 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.734223 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:18:12.734268 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:18:12.736032 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:18:12.738975 (+ 2943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:12.738978 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:12.738980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:12.739240 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:12.739559 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:12.739562 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:12.739564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:12.739664 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:12.740085 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:18:12.740088 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:18:12.740182 (+ 94us) client_negotiation.cc:769] Sending connection context
0812 03:18:12.740266 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 03:18:12.740400 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260812 03:18:12.741148 25441 log.cc:824] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442: Log is configured to *not* fsync() on all Append() calls
I20260812 03:18:12.741529 25717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.727002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34886 (local address 127.16.217.124:43355)
0812 03:18:12.727287 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.727297 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.727310 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:18:12.727346 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.727350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.727389 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.728557 (+ 1168us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.728666 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.729851 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.733096 (+ 3245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.733481 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.734021 (+ 540us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.736466 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:12.736929 (+ 463us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:12.736933 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:12.736959 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:12.738851 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.739364 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.739367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.739368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.739461 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.739769 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.739771 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.739773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.739983 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:12.741077 (+ 1094us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:12.741153 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 03:18:12.741339 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":226,"thread_start_us":81,"threads_started":1}
Aug 12 03:18:12 dist-test-slave-7nm7 krb5kdc[25293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504692, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.124@KRBTEST.COM for kudu/127.16.217.125@KRBTEST.COM
Aug 12 03:18:12 dist-test-slave-7nm7 krb5kdc[25293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:18:12.744613 25441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442: No bootstrap required, opened a new log
I20260812 03:18:12.747074 25560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e: Bootstrap starting.
I20260812 03:18:12.748040 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.738217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51360 (local address 127.16.217.125:46557)
0812 03:18:12.738365 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.738372 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.738387 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:18:12.738438 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.738442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.738492 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.738575 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.739095 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.739838 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.740900 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.741261 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.741341 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.743758 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:12.743762 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:12.743764 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:12.743790 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:12.745515 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.746013 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.746017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.746018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.746102 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.746462 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.746466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.746470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.746866 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:12.746951 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:12.747734 (+ 783us) server_negotiation.cc:299] Negotiation successful
0812 03:18:12.747866 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 03:18:12.748816 25713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.737955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:46557 (local address 127.0.0.1:51360)
0812 03:18:12.738087 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.738100 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:18:12.738182 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:18:12.738623 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:18:12.738628 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:18:12.738853 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:18:12.738970 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:12.738976 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.739962 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.739965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:12.740751 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:12.740767 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.741482 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.741485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:12.741578 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.742090 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:18:12.742103 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:18:12.743588 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:18:12.745647 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:12.745650 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:12.745652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:12.745893 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:12.746263 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:12.746266 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:12.746268 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:12.746355 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:12.747036 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:18:12.747047 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:18:12.747352 (+ 305us) client_negotiation.cc:769] Sending connection context
0812 03:18:12.748433 (+ 1081us) client_negotiation.cc:240] Negotiation successful
0812 03:18:12.748602 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 03:18:12.749296 25447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.738732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38716 (local address 127.16.217.126:36203)
0812 03:18:12.738839 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.738844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.738856 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:18:12.738892 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.738894 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.738931 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.739000 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.739814 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.740510 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.741589 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.741935 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.742015 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.745019 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:12.745026 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:12.745031 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:12.745060 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:12.747130 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.747652 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.747656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.747658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.747752 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.748079 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.748082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.748083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.748296 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:12.748469 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:12.749040 (+ 571us) server_negotiation.cc:299] Negotiation successful
0812 03:18:12.749159 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:18:12.749719 25701 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:18:12.750407 25560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e: Neither blocks nor log segments found. Creating new log.
I20260812 03:18:12.750701 25441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ee8da813f3489d9fd360126acad442" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } } peers { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } } peers { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } }
I20260812 03:18:12.750949 25441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43ee8da813f3489d9fd360126acad442, State: Initialized, Role: FOLLOWER
I20260812 03:18:12.751302 25441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ee8da813f3489d9fd360126acad442" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } } peers { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } } peers { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } }
I20260812 03:18:12.751382 25560 log.cc:824] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:18:12.751888 25721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ee8da813f3489d9fd360126acad442" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } } peers { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } } peers { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } } }
I20260812 03:18:12.752034 25721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:18:12.752894 25441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:18:12.754002 25560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e: No bootstrap required, opened a new log
Aug 12 03:18:12 dist-test-slave-7nm7 krb5kdc[25293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.125@KRBTEST.COM
I20260812 03:18:12.758841 25560 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ee8da813f3489d9fd360126acad442" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } } peers { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } } peers { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } }
I20260812 03:18:12.759114 25560 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bf4ef56b59f4589a3e8831077b6728e, State: Initialized, Role: FOLLOWER
Aug 12 03:18:12 dist-test-slave-7nm7 krb5kdc[25293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504692, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.124@KRBTEST.COM for kudu/127.16.217.124@KRBTEST.COM
I20260812 03:18:12.759498 25560 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ee8da813f3489d9fd360126acad442" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } } peers { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } } peers { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } }
I20260812 03:18:12.761056 25560 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:18:12.757853 25316 master_runner.cc:186] Error getting master registration for 127.16.217.125:46557: OK, instance_id { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" instance_seqno: 1786504692472506 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:18:12.762370 25316 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:18:12.762629 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.750985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51364 (local address 127.16.217.125:46557)
0812 03:18:12.751088 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.751092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.751268 (+ 176us) server_negotiation.cc:407] Connection header received
0812 03:18:12.751380 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.751383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.751433 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.751510 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.752393 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.753111 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.754256 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.754579 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.754658 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.757829 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:12.757834 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:12.757836 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:12.757864 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:12.759934 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.761006 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.761010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.761012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.761106 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.761499 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.761502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.761504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.761717 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:12.761871 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:12.762329 (+ 458us) server_negotiation.cc:299] Negotiation successful
0812 03:18:12.762464 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:18:12.763199 25717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.750373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34890 (local address 127.16.217.124:43355)
0812 03:18:12.750470 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.750474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.754476 (+ 4002us) server_negotiation.cc:407] Connection header received
0812 03:18:12.754651 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.754654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.754694 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.754742 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.755332 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.755992 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.757113 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.757434 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.757512 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.759846 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:12.759849 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:12.759852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:12.759872 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:12.761517 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.761943 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.761946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.761948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.762019 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.762355 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.762357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.762359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.762568 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:12.762644 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:12.762935 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 03:18:12.763049 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 03:18:12.763605 25713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.750220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:43355 (local address 127.0.0.1:34890)
0812 03:18:12.754396 (+ 4176us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:12.754415 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:18:12.754549 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:18:12.754893 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:18:12.754896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:18:12.755093 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:18:12.755200 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:12.755208 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.756260 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.756263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:12.756995 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:12.757009 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.757659 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.757662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:12.757739 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.758190 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:18:12.758205 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:18:12.759711 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:18:12.761631 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:12.761634 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:12.761636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:12.761848 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:12.762108 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:12.762111 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:12.762113 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:12.762209 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:12.762705 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:18:12.762707 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:18:12.762789 (+ 82us) client_negotiation.cc:769] Sending connection context
0812 03:18:12.763340 (+ 551us) client_negotiation.cc:240] Negotiation successful
0812 03:18:12.763476 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4074}
I20260812 03:18:12.764153 25746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ee8da813f3489d9fd360126acad442" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } } peers { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } } peers { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } } }
I20260812 03:18:12.764362 25746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:18:12.766845 25701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6: Bootstrap starting.
W20260812 03:18:12.767151 25744 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:18:12.767227 25744 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260812 03:18:12.767688 25769 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:18:12.767772 25769 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:18:12.769479 25701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6: Neither blocks nor log segments found. Creating new log.
I20260812 03:18:12.769694 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:18:12.770450 25701 log.cc:824] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6: Log is configured to *not* fsync() on all Append() calls
I20260812 03:18:12.769866 25450 master_runner.cc:186] Error getting master registration for 127.16.217.124:43355: OK, instance_id { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" instance_seqno: 1786504692709612 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:18:12.773952 25450 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:18:12.774163 25387 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "43ee8da813f3489d9fd360126acad442"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38716
I20260812 03:18:12.774451 25387 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 03:18:12.774585 25387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:18:12.776033 25387 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ee8da813f3489d9fd360126acad442" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } } peers { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } } peers { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } }
I20260812 03:18:12.777652 25387 leader_election.cc:288] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [CANDIDATE]: Term 1 election: Requested vote from peers 7bf4ef56b59f4589a3e8831077b6728e (127.16.217.125:46557), 17ab857126524c4ca09454c81a0643e6 (127.16.217.124:43355)
I20260812 03:18:12.777616 25515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "43ee8da813f3489d9fd360126acad442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7bf4ef56b59f4589a3e8831077b6728e"
I20260812 03:18:12.777864 25515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:18:12.779317 25515 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43ee8da813f3489d9fd360126acad442 in term 1.
I20260812 03:18:12.779783 25348 leader_election.cc:302] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43ee8da813f3489d9fd360126acad442, 7bf4ef56b59f4589a3e8831077b6728e; no voters:
I20260812 03:18:12.780093 25721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:18:12.780330 25721 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [term 1 LEADER]: Becoming Leader. State: Replica: 43ee8da813f3489d9fd360126acad442, State: Running, Role: LEADER
I20260812 03:18:12.780673 25721 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ee8da813f3489d9fd360126acad442" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } } peers { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } } peers { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } }
I20260812 03:18:12.780864 25701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6: No bootstrap required, opened a new log
I20260812 03:18:12.782582 25721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43ee8da813f3489d9fd360126acad442. Latest consensus state: current_term: 1 leader_uuid: "43ee8da813f3489d9fd360126acad442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ee8da813f3489d9fd360126acad442" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } } peers { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } } peers { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } } }
I20260812 03:18:12.782698 25721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:12.783321 25787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:18:12.784358 25787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:18:12.789359 25701 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ee8da813f3489d9fd360126acad442" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } } peers { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } } peers { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } }
I20260812 03:18:12.789937 25701 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ab857126524c4ca09454c81a0643e6, State: Initialized, Role: FOLLOWER
I20260812 03:18:12.790940 25701 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ee8da813f3489d9fd360126acad442" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } } peers { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } } peers { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } }
I20260812 03:18:12.792704 25515 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e [term 1 FOLLOWER]: Refusing update from remote peer 43ee8da813f3489d9fd360126acad442: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:18:12.793460 25721 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:18:12.798213 25746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43ee8da813f3489d9fd360126acad442. Latest consensus state: current_term: 1 leader_uuid: "43ee8da813f3489d9fd360126acad442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ee8da813f3489d9fd360126acad442" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } } peers { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } } peers { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } } }
I20260812 03:18:12.798390 25746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:18:12.799798 25657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "43ee8da813f3489d9fd360126acad442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "17ab857126524c4ca09454c81a0643e6"
I20260812 03:18:12.800041 25653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:18:12.800819 25701 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:18:12.801551 25653 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6 [term 1 FOLLOWER]: Refusing update from remote peer 43ee8da813f3489d9fd360126acad442: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:18:12.802372 25777 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:18:12.802951 25657 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43ee8da813f3489d9fd360126acad442 in term 1.
I20260812 03:18:12.808075 25798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43ee8da813f3489d9fd360126acad442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ee8da813f3489d9fd360126acad442" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } } peers { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } } peers { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } } }
I20260812 03:18:12.808291 25798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:18:12.808468 25798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43ee8da813f3489d9fd360126acad442. Latest consensus state: current_term: 1 leader_uuid: "43ee8da813f3489d9fd360126acad442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ee8da813f3489d9fd360126acad442" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } } peers { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } } peers { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } } }
I20260812 03:18:12.810896 25798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:18:12.815495 25717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.786721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34900 (local address 127.16.217.124:43355)
0812 03:18:12.786849 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.786854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.791095 (+ 4241us) server_negotiation.cc:407] Connection header received
0812 03:18:12.791839 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.791843 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.791900 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.791957 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.797200 (+ 5243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.798102 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.804200 (+ 6098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.804563 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.804658 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.811234 (+ 6576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:12.811239 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:12.811242 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:12.811278 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:12.813476 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.813954 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.813958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.813960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.814056 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.814465 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.814469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.814471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.814698 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:12.814847 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:12.815220 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 03:18:12.815345 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:18:12.819082 25777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "43ee8da813f3489d9fd360126acad442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ee8da813f3489d9fd360126acad442" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } } peers { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } } peers { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } } }
I20260812 03:18:12.819229 25777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:12.819346 25796 mvcc.cc:205] Tried to move back new op lower bound from 7317523221667676160 to 7317523221633232896. Current Snapshot: MvccSnapshot[applied={T|T < 7317523221667676160}]
I20260812 03:18:12.820643 25746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "43ee8da813f3489d9fd360126acad442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ee8da813f3489d9fd360126acad442" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } } peers { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } } peers { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } } }
I20260812 03:18:12.820782 25746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:18:12.823638 25721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "43ee8da813f3489d9fd360126acad442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ee8da813f3489d9fd360126acad442" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } } peers { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } } peers { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } } }
I20260812 03:18:12.823764 25721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442 [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:12.824940 25805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "43ee8da813f3489d9fd360126acad442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ee8da813f3489d9fd360126acad442" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 36203 } } peers { permanent_uuid: "7bf4ef56b59f4589a3e8831077b6728e" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 46557 } } peers { permanent_uuid: "17ab857126524c4ca09454c81a0643e6" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 43355 } } }
I20260812 03:18:12.825073 25805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:18:12.825724 25812 mvcc.cc:205] Tried to move back new op lower bound from 7317523221667676160 to 7317523221633232896. Current Snapshot: MvccSnapshot[applied={T|T < 7317523221667676160}]
W20260812 03:18:12.827551 25824 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260812 03:18:12.827775 25787 catalog_manager.cc:1347] Generated new cluster ID: 8ab88556f0354d15b6e49fde7546eb42
I20260812 03:18:12.827826 25787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:18:12.829212 25447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.787583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38722 (local address 127.16.217.126:36203)
0812 03:18:12.787698 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.787702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.794290 (+ 6588us) server_negotiation.cc:407] Connection header received
0812 03:18:12.795165 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.795169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.795227 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.795283 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.807887 (+ 12604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.808939 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.812961 (+ 4022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.813317 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.813390 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.821803 (+ 8413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:12.821807 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:12.821810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:12.821839 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:12.827020 (+ 5181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.827514 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.827518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.827519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.827617 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.827949 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.827953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.827954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.828180 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:12.828314 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:12.828927 (+ 613us) server_negotiation.cc:299] Negotiation successful
0812 03:18:12.829050 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:18:12.831288 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:12.788175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51368 (local address 127.16.217.125:46557)
0812 03:18:12.788303 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.788309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.799041 (+ 10732us) server_negotiation.cc:407] Connection header received
0812 03:18:12.799440 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.799446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.799505 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.799586 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.808399 (+ 8813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.809275 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.811690 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:12.812076 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:12.812172 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:12.819798 (+ 7626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:12.819802 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:12.819806 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:12.819836 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:12.821529 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.825916 (+ 4387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.825920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.825922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.826018 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:12.830307 (+ 4289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:12.830311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:12.830312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:12.830539 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:12.830674 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:12.831007 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 03:18:12.831132 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260812 03:18:12.840802 25824 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:18:12.846329 25787 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:18:12.847858 25787 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:18:12.860682 25787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43ee8da813f3489d9fd360126acad442: Generated new TSK 0
I20260812 03:18:12.861665 25787 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:18:12.893703 25717 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:12.890383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34912 (local address 127.16.217.124:43355)
0812 03:18:12.890520 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.890525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.892577 (+ 2052us) server_negotiation.cc:407] Connection header received
0812 03:18:12.892783 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.892787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.892843 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.892898 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.893557 (+ 659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34912: BlockingRecv error: recv got EOF from 127.0.0.1:34912 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 03:18:12.895267 25447 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:12.891243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38732 (local address 127.16.217.126:36203)
0812 03:18:12.891360 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.891365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.894355 (+ 2990us) server_negotiation.cc:407] Connection header received
0812 03:18:12.894530 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.894533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.894584 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.894636 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.895148 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38732: BlockingRecv error: recv got EOF from 127.0.0.1:38732 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 03:18:12.896857 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:12.891956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51372 (local address 127.16.217.125:46557)
0812 03:18:12.892071 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.892075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.895848 (+ 3773us) server_negotiation.cc:407] Connection header received
0812 03:18:12.896088 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.896092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.896154 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.896207 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.896704 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51372: BlockingRecv error: recv got EOF from 127.0.0.1:51372 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 03:18:12.901558 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:12.899949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51378 (local address 127.16.217.125:46557)
0812 03:18:12.900052 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.900056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.900562 (+ 506us) server_negotiation.cc:407] Connection header received
0812 03:18:12.900776 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.900780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.900854 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.900904 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.901466 (+ 562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51378: BlockingRecv error: recv got EOF from 127.0.0.1:51378 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 03:18:12.907059 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:12.905438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51382 (local address 127.16.217.125:46557)
0812 03:18:12.905542 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.905546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.906177 (+ 631us) server_negotiation.cc:407] Connection header received
0812 03:18:12.906382 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.906385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.906434 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.906483 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.906970 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51382: BlockingRecv error: recv got EOF from 127.0.0.1:51382 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 03:18:12.912762 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:12.911336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51396 (local address 127.16.217.125:46557)
0812 03:18:12.911433 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.911437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.911914 (+ 477us) server_negotiation.cc:407] Connection header received
0812 03:18:12.912089 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.912092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.912152 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.912201 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.912669 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51396: BlockingRecv error: recv got EOF from 127.0.0.1:51396 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 03:18:12.920562 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:12.919061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51404 (local address 127.16.217.125:46557)
0812 03:18:12.919205 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.919209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.919687 (+ 478us) server_negotiation.cc:407] Connection header received
0812 03:18:12.919869 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.919872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.919919 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.919968 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.920463 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51404: BlockingRecv error: recv got EOF from 127.0.0.1:51404 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260812 03:18:12.932695 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:12.931006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51406 (local address 127.16.217.125:46557)
0812 03:18:12.931119 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.931124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.931674 (+ 550us) server_negotiation.cc:407] Connection header received
0812 03:18:12.931869 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.931872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.931923 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.931975 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.932580 (+ 605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51406: BlockingRecv error: recv got EOF from 127.0.0.1:51406 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 03:18:12.953625 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:12.951709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51420 (local address 127.16.217.125:46557)
0812 03:18:12.951838 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.951843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.952445 (+ 602us) server_negotiation.cc:407] Connection header received
0812 03:18:12.952671 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.952688 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.952747 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.952809 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.953501 (+ 692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51420: BlockingRecv error: recv got EOF from 127.0.0.1:51420 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 03:18:12.989898 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:12.988589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51436 (local address 127.16.217.125:46557)
0812 03:18:12.988729 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:12.988734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:12.988990 (+ 256us) server_negotiation.cc:407] Connection header received
0812 03:18:12.989166 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:12.989171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:12.989224 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:12.989315 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:12.989778 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51436: BlockingRecv error: recv got EOF from 127.0.0.1:51436 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 03:18:13.057335 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:13.056027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51440 (local address 127.16.217.125:46557)
0812 03:18:13.056180 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:13.056195 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:13.056367 (+ 172us) server_negotiation.cc:407] Connection header received
0812 03:18:13.056545 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:13.056550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:13.056605 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:13.056703 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:13.057176 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51440: BlockingRecv error: recv got EOF from 127.0.0.1:51440 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 03:18:13.189124 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:13.187890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51448 (local address 127.16.217.125:46557)
0812 03:18:13.188072 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:13.188077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:13.188208 (+ 131us) server_negotiation.cc:407] Connection header received
0812 03:18:13.188396 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:13.188399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:13.188448 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:13.188540 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:13.189013 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51448: BlockingRecv error: recv got EOF from 127.0.0.1:51448 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260812 03:18:13.449170 25562 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:13.447749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51454 (local address 127.16.217.125:46557)
0812 03:18:13.447931 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:13.447936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:13.447952 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:18:13.448254 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:13.448261 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:13.448328 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:13.448465 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:13.449051 (+ 586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51454: BlockingRecv error: recv got EOF from 127.0.0.1:51454 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 03:18:13.769321 25769 catalog_manager.cc:1259] Loaded cluster ID: 8ab88556f0354d15b6e49fde7546eb42
I20260812 03:18:13.769402 25769 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e: loading cluster ID for follower catalog manager: success
I20260812 03:18:13.771061 25769 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e: acquiring CA information for follower catalog manager: success
I20260812 03:18:13.772167 25769 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7bf4ef56b59f4589a3e8831077b6728e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:18:13.842339 25824 catalog_manager.cc:1259] Loaded cluster ID: 8ab88556f0354d15b6e49fde7546eb42
I20260812 03:18:13.842429 25824 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6: loading cluster ID for follower catalog manager: success
I20260812 03:18:13.844053 25824 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6: acquiring CA information for follower catalog manager: success
I20260812 03:18:13.845182 25824 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 17ab857126524c4ca09454c81a0643e6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:18:13.967990 25952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:13.963780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51464 (local address 127.16.217.125:46557)
0812 03:18:13.964022 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:13.964026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:13.964044 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:18:13.964254 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:13.964260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:13.964411 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:13.964527 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:18:13.964791 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:13.965372 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:13.966331 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:13.966586 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:13.966675 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:13.966974 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:18:13.967452 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:18:13.967564 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:13.967706 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 03:18:13.967786 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":76,"threads_started":1}
I20260812 03:18:13.975406 25960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:13.971640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34918 (local address 127.16.217.124:43355)
0812 03:18:13.971857 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:13.971861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:13.971874 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:18:13.972036 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:13.972039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:13.972160 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:13.972278 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:18:13.972498 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:13.973046 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:13.973845 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:13.974075 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:13.974156 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:13.974505 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:18:13.974942 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:18:13.975044 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:13.975121 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 03:18:13.975170 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":70,"threads_started":1}
I20260812 03:18:13.982939 25952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:13.978457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51468 (local address 127.16.217.125:46557)
0812 03:18:13.978592 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:13.978596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:13.978895 (+ 299us) server_negotiation.cc:407] Connection header received
0812 03:18:13.979060 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:13.979063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:13.979126 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:13.979219 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:18:13.979484 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:13.980249 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:13.981593 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:13.981923 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:13.982026 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:13.982318 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:18:13.982408 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:18:13.982502 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:13.982696 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 03:18:13.982750 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:18:13.983505 25978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:13.978427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38738 (local address 127.16.217.126:36203)
0812 03:18:13.979163 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:13.979166 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:13.979178 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:18:13.979248 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:13.979250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:13.979352 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:13.979486 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:18:13.979857 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:13.980557 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:13.981541 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:13.981918 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:13.982031 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:13.982453 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:18:13.982831 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:18:13.982951 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:13.983305 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 03:18:13.983371 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":61,"threads_started":1}
I20260812 03:18:13.983543 25960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:13.978880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34926 (local address 127.16.217.124:43355)
0812 03:18:13.979020 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:13.979024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:13.979527 (+ 503us) server_negotiation.cc:407] Connection header received
0812 03:18:13.979642 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:13.979646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:13.979699 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:13.979788 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:18:13.980033 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:13.980820 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:13.981958 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:13.982401 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:13.982504 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:13.983054 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:18:13.983142 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:18:13.983233 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:13.983387 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 03:18:13.983436 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:18:13.990904 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 25316
I20260812 03:18:14.011960 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 25450
I20260812 03:18:14.026320 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 25572
2026-08-12T03:18:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3431 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:18:14 dist-test-slave-7nm7 krb5kdc[25989](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:18:14 dist-test-slave-7nm7 krb5kdc[25989](info): set up 2 sockets
Aug 12 03:18:14 dist-test-slave-7nm7 krb5kdc[25989](info): commencing operation
krb5kdc: starting...
W20260812 03:18:15.343551 17253 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/test-user.keytab.
Aug 12 03:18:15 dist-test-slave-7nm7 krb5kdc[25989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:18:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:18:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
Entry for principal kudu/127.16.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab.
I20260812 03:18:15.477202 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:46797
--webserver_interface=127.16.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:45995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:46797,127.16.217.125:41727,127.16.217.124:38901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:18:15.592947 26012 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:18:15.593312 26012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:18:15.593389 26012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:18:15.593472 26012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:18:15.597532 26012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:18:15.597623 26012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:18:15.597661 26012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:18:15.597718 26012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:18:15.597759 26012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:18:15.603384 26012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:45995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:46797,127.16.217.125:41727,127.16.217.124:38901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.126.keytab
--principal=kudu/127.16.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.126:46797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:18:15.604475 26012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:18:15.605530 26012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:18:15.614141 26022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:18:15.614169 26020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:18:15.614446 26012 server_base.cc:1034] running on GCE node
W20260812 03:18:15.614142 26026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:18:15.615098 26012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:18:15.616207 26012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:18:15.617398 26012 hybrid_clock.cc:648] HybridClock initialized: now 1786504695617373 us; error 33 us; skew 500 ppm
Aug 12 03:18:15 dist-test-slave-7nm7 krb5kdc[25989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504695, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:18:15.620602 26012 init.cc:377] Logged in from keytab as kudu/127.16.217.126@KRBTEST.COM (short username kudu)
I20260812 03:18:15.623441 26012 webserver.cc:460] Webserver started at http://127.16.217.126:35163/ using document root <none> and password file <none>
I20260812 03:18:15.624053 26012 fs_manager.cc:359] Metadata directory not provided
I20260812 03:18:15.624120 26012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:18:15.624349 26012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:18:15.626562 26012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-0/data/instance:
uuid: "4c5da44e4769412ba16d58d9de1f6777"
format_stamp: "Formatted at 2026-08-12 03:18:15 on dist-test-slave-7nm7"
server_key: "107aea12c453a0bf79f96d5ab53d158b"
server_key_iv: "7517df5b9ad5626d4b803f6017127b6e"
server_key_version: "encryptionkey@0"
I20260812 03:18:15.627156 26012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-0/wal/instance:
uuid: "4c5da44e4769412ba16d58d9de1f6777"
format_stamp: "Formatted at 2026-08-12 03:18:15 on dist-test-slave-7nm7"
server_key: "107aea12c453a0bf79f96d5ab53d158b"
server_key_iv: "7517df5b9ad5626d4b803f6017127b6e"
server_key_version: "encryptionkey@0"
I20260812 03:18:15.630971 26012 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 03:18:15.633606 26039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:18:15.635035 26012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:18:15.635187 26012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-0/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-0/wal
uuid: "4c5da44e4769412ba16d58d9de1f6777"
format_stamp: "Formatted at 2026-08-12 03:18:15 on dist-test-slave-7nm7"
server_key: "107aea12c453a0bf79f96d5ab53d158b"
server_key_iv: "7517df5b9ad5626d4b803f6017127b6e"
server_key_version: "encryptionkey@0"
I20260812 03:18:15.635306 26012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:18:15.655659 26012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:18:15.658979 26012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:18:15.659209 26012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:18:15.666714 26012 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.126:46797
I20260812 03:18:15.667013 26135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.126:46797 every 8 connection(s)
I20260812 03:18:15.667742 26012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-0/data/info.pb
I20260812 03:18:15.670800 26136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:18:15.674595 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 26012
I20260812 03:18:15.674736 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-0/wal/instance
I20260812 03:18:15.675065 17253 external_mini_cluster.cc:1428] Setting key 3a50c038ee798a9553d347709f173fa1
I20260812 03:18:15.674906 26136 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.16.217.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.125@KRBTEST.COM" created.
Aug 12 03:18:15 dist-test-slave-7nm7 krb5kdc[25989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504695, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:18:15.694028 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.677677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:46797 (local address 127.0.0.1:55916)
0812 03:18:15.678287 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:15.678352 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 03:18:15.679447 (+ 1095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:18:15.680278 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:18:15.680282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:18:15.680621 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:18:15.680882 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:15.680894 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:15.682734 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:15.682737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:15.683433 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:15.683442 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:15.684448 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:15.684452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:15.684535 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:15.685248 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:18:15.685264 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:18:15.688876 (+ 3612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:18:15.691003 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:15.691011 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:15.691020 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:15.691257 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:15.691640 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:15.691643 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:15.691644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:15.691769 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:15.692642 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:18:15.692648 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:18:15.692777 (+ 129us) client_negotiation.cc:769] Sending connection context
0812 03:18:15.692928 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 03:18:15.693204 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":1,"thread_start_us":148,"threads_started":1}
I20260812 03:18:15.694036 26145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.677677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55916 (local address 127.16.217.126:46797)
0812 03:18:15.678264 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:15.678283 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:15.678506 (+ 223us) server_negotiation.cc:407] Connection header received
0812 03:18:15.679789 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:15.679799 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:15.680069 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:15.680276 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:15.681062 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:15.682614 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:15.683567 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:15.684254 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:15.684467 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:15.689076 (+ 4609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:15.689082 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:15.689093 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:15.689116 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:15.690855 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:15.691373 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:15.691382 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:15.691387 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:15.691535 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:15.691867 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:15.691873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:15.691878 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:15.692487 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:15.692644 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:15.693048 (+ 404us) server_negotiation.cc:299] Negotiation successful
0812 03:18:15.693291 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":442,"spinlock_wait_cycles":53504,"thread_start_us":175,"threads_started":1}
I20260812 03:18:15.696873 26136 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:18:15.697678 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.697409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:41727 (local address 127.0.0.1:60812)
0812 03:18:15.697538 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:15.697615 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:41727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 03:18:15.697911 26042 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.217.125:41727: connect: Connection refused (error 111)
I20260812 03:18:15.698976 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.698752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:41727 (local address 127.0.0.1:60828)
0812 03:18:15.698859 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:15.698907 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:41727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 03:18:15.699131 26136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.125:41727: Network error: Client connection negotiation failed: client connection to 127.16.217.125:41727: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.125.keytab.
Entry for principal kudu/127.16.217.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.125.keytab.
I20260812 03:18:15.705015 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.125:41727
--webserver_interface=127.16.217.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.125.keytab
--principal=kudu/127.16.217.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:45995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:46797,127.16.217.125:41727,127.16.217.124:38901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:18:15.748416 26136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } attempt: 1
I20260812 03:18:15.749382 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.749054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:41727 (local address 127.0.0.1:60834)
0812 03:18:15.749230 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:15.749303 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:41727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 03:18:15.750168 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.749931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:41727 (local address 127.0.0.1:60846)
0812 03:18:15.750045 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:15.750113 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:41727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 03:18:15.750331 26136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.125:41727: Network error: Client connection negotiation failed: client connection to 127.16.217.125:41727: connect: Connection refused (error 111)
I20260812 03:18:15.818519 26136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } attempt: 2
I20260812 03:18:15.819594 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.819283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:41727 (local address 127.0.0.1:60856)
0812 03:18:15.819447 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:15.819518 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:41727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 03:18:15.820525 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.820180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:41727 (local address 127.0.0.1:60872)
0812 03:18:15.820396 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:15.820462 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.125:41727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 03:18:15.820668 26136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.125:41727: Network error: Client connection negotiation failed: client connection to 127.16.217.125:41727: connect: Connection refused (error 111)
W20260812 03:18:15.822506 26150 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:18:15.822805 26150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:18:15.822865 26150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:18:15.822944 26150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:18:15.827020 26150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:18:15.827093 26150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:18:15.827138 26150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:18:15.827178 26150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:18:15.827214 26150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:18:15.832511 26150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:45995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:46797,127.16.217.125:41727,127.16.217.124:38901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.125.keytab
--principal=kudu/127.16.217.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.125:41727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.217.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:18:15.833578 26150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:18:15.834654 26150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:18:15.843689 26158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:18:15.843742 26157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:18:15.843732 26150 server_base.cc:1034] running on GCE node
W20260812 03:18:15.843689 26162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:18:15.844480 26150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:18:15.845621 26150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:18:15.846817 26150 hybrid_clock.cc:648] HybridClock initialized: now 1786504695846788 us; error 42 us; skew 500 ppm
Aug 12 03:18:15 dist-test-slave-7nm7 krb5kdc[25989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504695, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:18:15.849506 26150 init.cc:377] Logged in from keytab as kudu/127.16.217.125@KRBTEST.COM (short username kudu)
I20260812 03:18:15.850440 26150 webserver.cc:460] Webserver started at http://127.16.217.125:41675/ using document root <none> and password file <none>
I20260812 03:18:15.850960 26150 fs_manager.cc:359] Metadata directory not provided
I20260812 03:18:15.851016 26150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:18:15.851179 26150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:18:15.853132 26150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-1/data/instance:
uuid: "0258d5c267ad4bf98506b8a768ccf9e5"
format_stamp: "Formatted at 2026-08-12 03:18:15 on dist-test-slave-7nm7"
server_key: "4178c1eb8f4e8f160695f917eb0603fd"
server_key_iv: "3fbe23192b7bcfb097b5c1cede914e6e"
server_key_version: "encryptionkey@0"
I20260812 03:18:15.853691 26150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-1/wal/instance:
uuid: "0258d5c267ad4bf98506b8a768ccf9e5"
format_stamp: "Formatted at 2026-08-12 03:18:15 on dist-test-slave-7nm7"
server_key: "4178c1eb8f4e8f160695f917eb0603fd"
server_key_iv: "3fbe23192b7bcfb097b5c1cede914e6e"
server_key_version: "encryptionkey@0"
I20260812 03:18:15.857513 26150 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:18:15.859972 26176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:18:15.861259 26150 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:18:15.861400 26150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-1/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-1/wal
uuid: "0258d5c267ad4bf98506b8a768ccf9e5"
format_stamp: "Formatted at 2026-08-12 03:18:15 on dist-test-slave-7nm7"
server_key: "4178c1eb8f4e8f160695f917eb0603fd"
server_key_iv: "3fbe23192b7bcfb097b5c1cede914e6e"
server_key_version: "encryptionkey@0"
I20260812 03:18:15.861508 26150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:18:15.881295 26150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:18:15.884313 26150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:18:15.884517 26150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:18:15.891932 26150 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.125:41727
I20260812 03:18:15.892230 26257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.125:41727 every 8 connection(s)
I20260812 03:18:15.893064 26150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-1/data/info.pb
I20260812 03:18:15.895866 26258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:18:15.899566 26258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:18:15.902949 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 26150
I20260812 03:18:15.903100 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-1/wal/instance
I20260812 03:18:15.903393 17253 external_mini_cluster.cc:1428] Setting key 6b52ebc1a564a53c2cbfd33dc12c29d7
Aug 12 03:18:15 dist-test-slave-7nm7 krb5kdc[25989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504695, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.125@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:18:15.911855 26136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } attempt: 3
WARNING: no policy specified for kudu/127.16.217.124@KRBTEST.COM; defaulting to no policy
I20260812 03:18:15.915047 26145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.901386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55922 (local address 127.16.217.126:46797)
0812 03:18:15.901523 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:15.901526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:15.902623 (+ 1097us) server_negotiation.cc:407] Connection header received
0812 03:18:15.903861 (+ 1238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:15.903865 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:15.903937 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:15.904049 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:15.905253 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:15.905819 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:15.907166 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:15.907411 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:15.907530 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:15.910521 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:15.910541 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:15.910547 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:15.910580 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:15.912549 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:15.913107 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:15.913110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:15.913112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:15.913201 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:15.913649 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:15.913652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:15.913654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:15.913850 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:15.913977 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:15.914687 (+ 710us) server_negotiation.cc:299] Negotiation successful
0812 03:18:15.914843 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.217.124@KRBTEST.COM" created.
I20260812 03:18:15.915879 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.901700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:46797 (local address 127.0.0.1:55922)
0812 03:18:15.902423 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:15.902502 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0812 03:18:15.903630 (+ 1128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:18:15.904228 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:18:15.904235 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:18:15.904606 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:18:15.905091 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:15.905106 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:15.905971 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:15.905975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:15.907024 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:15.907034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:15.907575 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:15.907579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:15.907751 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:15.908375 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:18:15.908393 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:18:15.910335 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:18:15.912671 (+ 2336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:15.912678 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:15.912698 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:15.912997 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:15.913298 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:15.913302 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:15.913304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:15.913519 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:15.913986 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:18:15.913992 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:18:15.914359 (+ 367us) client_negotiation.cc:769] Sending connection context
0812 03:18:15.914535 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 03:18:15.914806 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":141,"threads_started":1}
I20260812 03:18:15.917429 26258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:18:15 dist-test-slave-7nm7 krb5kdc[25989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504695, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for kudu/127.16.217.125@KRBTEST.COM
Aug 12 03:18:15 dist-test-slave-7nm7 krb5kdc[25989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504695, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.125@KRBTEST.COM for kudu/127.16.217.125@KRBTEST.COM
I20260812 03:18:15.925113 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.912399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:41727 (local address 127.0.0.1:60886)
0812 03:18:15.912610 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:15.912624 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:18:15.912818 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:18:15.913971 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:18:15.913974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:18:15.914315 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:18:15.914909 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:15.914919 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:15.916519 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:15.916523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:15.917163 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:15.917173 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:15.917911 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:15.917914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:15.918038 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:15.918655 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:18:15.918675 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:18:15.920123 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:18:15.923160 (+ 3037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:15.923164 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:15.923166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:15.923372 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:15.923698 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:15.923701 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:15.923703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:15.923787 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:15.924253 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:18:15.924256 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:18:15.924363 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 03:18:15.924491 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 03:18:15.924641 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260812 03:18:15.925827 26265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.913008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60886 (local address 127.16.217.125:41727)
0812 03:18:15.913555 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:15.913574 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:15.913601 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:18:15.913643 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:15.913648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:15.913836 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:15.913974 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:15.915093 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:15.916418 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:15.917292 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:15.917778 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:15.917915 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:15.920257 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:15.920267 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:15.920281 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:15.920312 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:15.923037 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:15.923500 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:15.923508 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:15.923510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:15.923597 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:15.923882 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:15.923887 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:15.923889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:15.924141 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:15.924287 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:15.924615 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 03:18:15.924795 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":144,"threads_started":1}
I20260812 03:18:15.927935 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.917997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:41727 (local address 127.0.0.1:60896)
0812 03:18:15.918665 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:15.918680 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:18:15.918747 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:18:15.919137 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:18:15.919140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:18:15.919325 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:18:15.919447 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:15.919455 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:15.920686 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:15.920692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:15.921248 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:15.921257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:15.921970 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:15.921974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:15.922063 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:15.922497 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:18:15.922527 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:18:15.924202 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:18:15.926378 (+ 2176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:15.926381 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:15.926383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:15.926650 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:15.926962 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:15.926965 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:15.926966 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:15.927062 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:15.927391 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:18:15.927394 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:18:15.927485 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 03:18:15.927612 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 03:18:15.927755 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189}
I20260812 03:18:15.927937 26136 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:18:15.928054 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.918259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60896 (local address 127.16.217.125:41727)
0812 03:18:15.918925 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:15.918932 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:15.918940 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:18:15.918961 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:15.918963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:15.918992 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:15.919127 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:15.919593 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:15.920517 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:15.921377 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:15.921816 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:15.921963 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:15.924347 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:15.924350 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:15.924353 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:15.924376 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:15.926220 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:15.926764 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:15.926770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:15.926773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:15.926858 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:15.927149 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:15.927151 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:15.927153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:15.927300 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:15.927373 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:15.927684 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 03:18:15.927808 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":619,"thread_start_us":158,"threads_started":1}
I20260812 03:18:15.928797 26258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:18:15.929179 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.928496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:38901 (local address 127.0.0.1:42474)
0812 03:18:15.928686 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:15.929103 (+ 417us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260812 03:18:15.929531 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.929244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:38901 (local address 127.0.0.1:42490)
0812 03:18:15.929394 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:15.929467 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260812 03:18:15.929754 26180 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
I20260812 03:18:15.929884 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.929619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:38901 (local address 127.0.0.1:42492)
0812 03:18:15.929776 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:15.929832 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260812 03:18:15.930025 26136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.124:38901: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
I20260812 03:18:15.930694 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.930499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:38901 (local address 127.0.0.1:42500)
0812 03:18:15.930602 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:15.930646 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 03:18:15.930851 26258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.124:38901: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.217.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.124.keytab.
Entry for principal kudu/127.16.217.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.124.keytab.
I20260812 03:18:15.935240 17253 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
/tmp/dist-test-taskm8e4iX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.124:38901
--webserver_interface=127.16.217.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.124.keytab
--principal=kudu/127.16.217.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.217.84:45995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.217.126:46797,127.16.217.125:41727,127.16.217.124:38901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:18:15.961207 26136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } attempt: 1
I20260812 03:18:15.962150 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.961832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:38901 (local address 127.0.0.1:42512)
0812 03:18:15.961994 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:15.962067 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 03:18:15.962888 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.962669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:38901 (local address 127.0.0.1:42524)
0812 03:18:15.962784 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:15.962834 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 03:18:15.963014 26136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.124:38901: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
I20260812 03:18:15.980142 26258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } attempt: 1
I20260812 03:18:15.981031 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.980735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:38901 (local address 127.0.0.1:42530)
0812 03:18:15.980887 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:15.980950 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 03:18:15.981772 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:15.981541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:38901 (local address 127.0.0.1:42542)
0812 03:18:15.981664 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:15.981715 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 03:18:15.981899 26258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.124:38901: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
I20260812 03:18:16.038223 26136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } attempt: 2
I20260812 03:18:16.039247 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.038924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:38901 (local address 127.0.0.1:42548)
0812 03:18:16.039103 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:16.039169 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 03:18:16.040006 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.039790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:38901 (local address 127.0.0.1:42562)
0812 03:18:16.039900 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:16.039952 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 03:18:16.040143 26136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.124:38901: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
I20260812 03:18:16.050077 26258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } attempt: 2
I20260812 03:18:16.050981 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.050707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:38901 (local address 127.0.0.1:42566)
0812 03:18:16.050847 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:16.050910 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 03:18:16.051735 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.051511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:38901 (local address 127.0.0.1:42574)
0812 03:18:16.051631 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:16.051680 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 03:18:16.051862 26258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.217.124:38901: Network error: Client connection negotiation failed: client connection to 127.16.217.124:38901: connect: Connection refused (error 111)
W20260812 03:18:16.052462 26271 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:18:16.052775 26271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:18:16.052835 26271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:18:16.052914 26271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:18:16.057385 26271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:18:16.057466 26271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:18:16.057518 26271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:18:16.057559 26271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:18:16.057595 26271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:18:16.063174 26271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.217.84:45995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.217.126:46797,127.16.217.125:41727,127.16.217.124:38901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/krb5kdc/kudu_127.16.217.124.keytab
--principal=kudu/127.16.217.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.217.124:38901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.217.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:18:16.064283 26271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:18:16.065519 26271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:18:16.074388 26278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:18:16.074420 26280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:18:16.074659 26271 server_base.cc:1034] running on GCE node
W20260812 03:18:16.074370 26283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:18:16.075261 26271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:18:16.076329 26271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:18:16.077531 26271 hybrid_clock.cc:648] HybridClock initialized: now 1786504696077515 us; error 28 us; skew 500 ppm
Aug 12 03:18:16 dist-test-slave-7nm7 krb5kdc[25989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504696, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:18:16.080332 26271 init.cc:377] Logged in from keytab as kudu/127.16.217.124@KRBTEST.COM (short username kudu)
I20260812 03:18:16.081182 26271 webserver.cc:460] Webserver started at http://127.16.217.124:34449/ using document root <none> and password file <none>
I20260812 03:18:16.081741 26271 fs_manager.cc:359] Metadata directory not provided
I20260812 03:18:16.081835 26271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:18:16.082062 26271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:18:16.084110 26271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-2/data/instance:
uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79"
format_stamp: "Formatted at 2026-08-12 03:18:16 on dist-test-slave-7nm7"
server_key: "adbf164e95e04f0240e4d11a093c7808"
server_key_iv: "d63b19b2bf043716d31fc3bb4ca92745"
server_key_version: "encryptionkey@0"
I20260812 03:18:16.084662 26271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-2/wal/instance:
uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79"
format_stamp: "Formatted at 2026-08-12 03:18:16 on dist-test-slave-7nm7"
server_key: "adbf164e95e04f0240e4d11a093c7808"
server_key_iv: "d63b19b2bf043716d31fc3bb4ca92745"
server_key_version: "encryptionkey@0"
I20260812 03:18:16.088464 26271 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260812 03:18:16.091042 26295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:18:16.092264 26271 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 03:18:16.092420 26271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-2/data,/tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-2/wal
uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79"
format_stamp: "Formatted at 2026-08-12 03:18:16 on dist-test-slave-7nm7"
server_key: "adbf164e95e04f0240e4d11a093c7808"
server_key_iv: "d63b19b2bf043716d31fc3bb4ca92745"
server_key_version: "encryptionkey@0"
I20260812 03:18:16.092536 26271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:18:16.113512 26271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:18:16.117041 26271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:18:16.117273 26271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:18:16.124670 26271 rpc_server.cc:307] RPC server started. Bound to: 127.16.217.124:38901
I20260812 03:18:16.125109 26387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.217.124:38901 every 8 connection(s)
I20260812 03:18:16.126307 26271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-2/data/info.pb
I20260812 03:18:16.129225 26388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:18:16.132651 17253 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu as pid 26271
I20260812 03:18:16.132803 17253 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm8e4iX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504382969880-17253-0/minicluster-data/master-2/wal/instance
I20260812 03:18:16.133124 17253 external_mini_cluster.cc:1428] Setting key 87953c64bfca65286acefb3023165222
I20260812 03:18:16.133234 26388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:18:16.139307 26136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } attempt: 3
Aug 12 03:18:16 dist-test-slave-7nm7 krb5kdc[25989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504696, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.124@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
Aug 12 03:18:16 dist-test-slave-7nm7 krb5kdc[25989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.124@KRBTEST.COM
I20260812 03:18:16.143029 26258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } attempt: 3
Aug 12 03:18:16 dist-test-slave-7nm7 krb5kdc[25989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504695, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.126@KRBTEST.COM for kudu/127.16.217.124@KRBTEST.COM
I20260812 03:18:16.146507 26145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.135313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55928 (local address 127.16.217.126:46797)
0812 03:18:16.135437 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.135442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.135678 (+ 236us) server_negotiation.cc:407] Connection header received
0812 03:18:16.136199 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.136201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.136245 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.136313 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.137261 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.137742 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.138891 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.139165 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.139243 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:16.141848 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:16.141852 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:16.141855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:16.141911 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:16.143974 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.144600 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.144605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.144609 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.144703 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.145155 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.145159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.145161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.145352 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:16.145466 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:16.146154 (+ 688us) server_negotiation.cc:299] Negotiation successful
0812 03:18:16.146324 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:18:16.147841 26402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.135152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.126:46797 (local address 127.0.0.1:55928)
0812 03:18:16.135526 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:16.135578 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 03:18:16.135985 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:18:16.136389 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:18:16.136394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:18:16.136829 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:18:16.137104 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:16.137131 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.137857 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.137860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:16.138775 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:16.138784 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.139299 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.139303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:16.139472 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:16.140098 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:18:16.140132 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:18:16.141699 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:18:16.144093 (+ 2394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:16.144103 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:16.144113 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:16.144486 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:16.144848 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:16.144852 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:16.144855 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:16.145038 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:16.145467 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:18:16.145475 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:18:16.145842 (+ 367us) client_negotiation.cc:769] Sending connection context
0812 03:18:16.146725 (+ 883us) client_negotiation.cc:240] Negotiation successful
0812 03:18:16.146998 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"spinlock_wait_cycles":896,"thread_start_us":103,"threads_started":1}
I20260812 03:18:16.148423 26400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.134573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42576 (local address 127.16.217.124:38901)
0812 03:18:16.134969 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.134982 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.135014 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:18:16.135985 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.136008 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.136244 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.136434 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.137135 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.138180 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.139042 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.139480 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.139573 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:16.142923 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:16.142929 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:16.142942 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:16.142967 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:16.145797 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.146447 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.146455 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.146457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.146541 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.146869 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.146872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.146874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.147216 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:16.147433 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:16.148115 (+ 682us) server_negotiation.cc:299] Negotiation successful
0812 03:18:16.148247 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"spinlock_wait_cycles":4992,"thread_start_us":113,"threads_started":1}
I20260812 03:18:16.149746 26388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:18:16 dist-test-slave-7nm7 krb5kdc[25989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504695, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.125@KRBTEST.COM for kudu/127.16.217.124@KRBTEST.COM
I20260812 03:18:16.152560 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.139701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:38901 (local address 127.0.0.1:42586)
0812 03:18:16.140096 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:16.140110 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:18:16.140188 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:18:16.140497 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:18:16.140501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:18:16.140677 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:18:16.140746 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:16.140751 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.142880 (+ 2129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.142885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:16.143633 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:16.143644 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.144525 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.144531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:16.144623 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:16.145069 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:18:16.145084 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:18:16.146541 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:18:16.150963 (+ 4422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:16.150969 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:16.150974 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:16.151233 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:16.151573 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:16.151576 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:16.151578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:16.151673 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:16.152095 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:18:16.152098 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:18:16.152187 (+ 89us) client_negotiation.cc:769] Sending connection context
0812 03:18:16.152257 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 03:18:16.152391 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316}
I20260812 03:18:16.153085 26404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.139829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42586 (local address 127.16.217.124:38901)
0812 03:18:16.140204 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.140218 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.140234 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:18:16.140271 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.140275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.140321 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.140483 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.141858 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.142696 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.143765 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.144137 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.144540 (+ 403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:16.147314 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:16.147320 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:16.147323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:16.147344 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:16.150837 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.151353 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.151358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.151360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.151451 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.151792 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.151795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.151796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.151984 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:16.152731 (+ 747us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:16.152804 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 03:18:16.152910 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":129,"threads_started":1}
I20260812 03:18:16.156584 26136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777: Bootstrap starting.
Aug 12 03:18:16 dist-test-slave-7nm7 krb5kdc[25989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504696, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.124@KRBTEST.COM for kudu/127.16.217.125@KRBTEST.COM
Aug 12 03:18:16 dist-test-slave-7nm7 krb5kdc[25989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.126@KRBTEST.COM
I20260812 03:18:16.158816 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.143480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:38901 (local address 127.0.0.1:42602)
0812 03:18:16.143649 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:16.143662 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:18:16.143742 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:18:16.144399 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:18:16.144402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:18:16.144592 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:18:16.144703 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:16.144728 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.147658 (+ 2930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.147662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:16.148423 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:16.148437 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.149033 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.149037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:16.149145 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:16.149603 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:18:16.149618 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:18:16.151079 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:18:16.157054 (+ 5975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:16.157058 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:16.157060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:16.157338 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:16.157685 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:16.157689 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:16.157691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:16.157789 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:16.158322 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:18:16.158325 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:18:16.158423 (+ 98us) client_negotiation.cc:769] Sending connection context
0812 03:18:16.158525 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 03:18:16.158665 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260812 03:18:16.159433 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.143875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42602 (local address 127.16.217.124:38901)
0812 03:18:16.144158 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.144169 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.144184 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:18:16.144222 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.144224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.144269 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.144367 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.144840 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.146662 (+ 1822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.148592 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.148906 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.149017 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:16.151229 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:16.151233 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:16.151236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:16.151262 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:16.156926 (+ 5664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.157473 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.157476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.157478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.157569 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.157947 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.157951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.157954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.158171 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:16.158315 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:16.159138 (+ 823us) server_negotiation.cc:299] Negotiation successful
0812 03:18:16.159251 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":110,"threads_started":1}
I20260812 03:18:16.159554 26136 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777: Neither blocks nor log segments found. Creating new log.
I20260812 03:18:16.160517 26136 log.cc:824] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777: Log is configured to *not* fsync() on all Append() calls
I20260812 03:18:16.162647 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.150481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60912 (local address 127.16.217.125:41727)
0812 03:18:16.150639 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.150645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.150660 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:18:16.150868 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.150871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.150918 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.150972 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.151572 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.152319 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.153350 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.153702 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.153804 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:16.158907 (+ 5103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:16.158911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:16.158913 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:16.158943 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:16.160695 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.161189 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.161192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.161194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.161284 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.161624 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.161627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.161629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.161865 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:16.161935 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:16.162355 (+ 420us) server_negotiation.cc:299] Negotiation successful
0812 03:18:16.162479 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 03:18:16.163064 26136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777: No bootstrap required, opened a new log
I20260812 03:18:16.163090 26402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.150263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.125:41727 (local address 127.0.0.1:60912)
0812 03:18:16.150596 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:16.150609 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:18:16.150694 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:18:16.151129 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:18:16.151132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:18:16.151299 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:18:16.151414 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:16.151421 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.152441 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.152444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:16.153209 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:16.153220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.153911 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.153913 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:16.154004 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:16.154485 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:18:16.154499 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:18:16.158772 (+ 4273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:18:16.160823 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:16.160826 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:16.160827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:16.161081 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:16.161381 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:16.161387 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:16.161392 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:16.161494 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:16.162011 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:18:16.162014 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:18:16.162105 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 03:18:16.162826 (+ 721us) client_negotiation.cc:240] Negotiation successful
0812 03:18:16.162956 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265}
I20260812 03:18:16.163514 26258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5: Bootstrap starting.
I20260812 03:18:16.163763 26388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:18:16.163961 26145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.151410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55940 (local address 127.16.217.126:46797)
0812 03:18:16.151818 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.151825 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.152005 (+ 180us) server_negotiation.cc:407] Connection header received
0812 03:18:16.152461 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.152468 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.152519 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.152575 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.153579 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.154298 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.155360 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.155570 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.155649 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:16.160239 (+ 4590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:16.160242 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:16.160245 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:16.160269 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:16.162043 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.162571 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.162575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.162576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.162661 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.162979 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.162982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.162984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.163200 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:16.163347 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:16.163714 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 03:18:16.163831 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:18:16.166534 26258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5: Neither blocks nor log segments found. Creating new log.
I20260812 03:18:16.167505 26258 log.cc:824] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5: Log is configured to *not* fsync() on all Append() calls
I20260812 03:18:16.167443 26136 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c5da44e4769412ba16d58d9de1f6777" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } } peers { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } } peers { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } }
I20260812 03:18:16.167688 26136 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c5da44e4769412ba16d58d9de1f6777, State: Initialized, Role: FOLLOWER
I20260812 03:18:16.168012 26136 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c5da44e4769412ba16d58d9de1f6777" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } } peers { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } } peers { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } }
I20260812 03:18:16.169399 26136 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:18:16.170702 26258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5: No bootstrap required, opened a new log
Aug 12 03:18:16 dist-test-slave-7nm7 krb5kdc[25989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504696, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.217.124@KRBTEST.COM for kudu/127.16.217.124@KRBTEST.COM
Aug 12 03:18:16 dist-test-slave-7nm7 krb5kdc[25989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.217.125@KRBTEST.COM
I20260812 03:18:16.174566 26423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c5da44e4769412ba16d58d9de1f6777" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } } peers { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } } peers { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } } }
I20260812 03:18:16.174712 26423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:18:16.175595 26402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.164199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.217.124:38901 (local address 127.0.0.1:42606)
0812 03:18:16.164635 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 03:18:16.164650 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:18:16.164794 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:18:16.166060 (+ 1266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:18:16.166063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:18:16.166258 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:18:16.166370 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:16.166379 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.167373 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.167376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:16.168119 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:18:16.168133 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.168794 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.168797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:18:16.168866 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:16.169699 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:18:16.169713 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:18:16.171265 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:18:16.173161 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:16.173164 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:16.173166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:16.173385 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:16.174517 (+ 1132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:18:16.174520 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:18:16.174522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:18:16.174632 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:18:16.175090 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:18:16.175095 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:18:16.175174 (+ 79us) client_negotiation.cc:769] Sending connection context
0812 03:18:16.175264 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 03:18:16.175401 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":112,"threads_started":1}
I20260812 03:18:16.176126 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.164504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42606 (local address 127.16.217.124:38901)
0812 03:18:16.165835 (+ 1331us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.165839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.165851 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:18:16.165883 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.165886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.165924 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.166001 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.166503 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.167243 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.168245 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.168591 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.168694 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:16.171399 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:16.171402 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:16.171405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:16.171427 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:16.173056 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.173511 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.173515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.173518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.174425 (+ 907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.174741 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.174745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.174746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.174969 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:16.175058 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:16.175866 (+ 808us) server_negotiation.cc:299] Negotiation successful
0812 03:18:16.175988 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1269}
I20260812 03:18:16.176586 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.165655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60922 (local address 127.16.217.125:41727)
0812 03:18:16.165754 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.165758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.166289 (+ 531us) server_negotiation.cc:407] Connection header received
0812 03:18:16.166454 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.166458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.166499 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.166553 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.167392 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.168078 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.169121 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.169443 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.169567 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:16.172505 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:16.172508 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:16.172510 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:16.172533 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:16.174280 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.174920 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.174923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.174925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.175010 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.175488 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.175492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.175493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.175735 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:16.175913 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:16.176274 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 03:18:16.176401 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 03:18:16.178723 26437 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:18:16.178792 26437 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:18:16.179795 17253 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:18:16.180517 26388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79: Bootstrap starting.
I20260812 03:18:16.181034 26084 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "4c5da44e4769412ba16d58d9de1f6777"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55940
I20260812 03:18:16.181268 26084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 03:18:16.181380 26084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:18:16.182780 26084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c5da44e4769412ba16d58d9de1f6777" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } } peers { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } } peers { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } }
I20260812 03:18:16.183192 26388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79: Neither blocks nor log segments found. Creating new log.
I20260812 03:18:16.183777 26084 leader_election.cc:288] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [CANDIDATE]: Term 1 election: Requested vote from peers 0258d5c267ad4bf98506b8a768ccf9e5 (127.16.217.125:41727), cee10ef2e11c4f1f8dd4f873d37e6c79 (127.16.217.124:38901)
I20260812 03:18:16.175874 26012 master_runner.cc:186] Error getting master registration for 127.16.217.125:41727: OK, instance_id { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" instance_seqno: 1786504695889846 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:18:16.184211 26012 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:18:16.184037 26388 log.cc:824] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79: Log is configured to *not* fsync() on all Append() calls
I20260812 03:18:16.186580 26258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c5da44e4769412ba16d58d9de1f6777" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } } peers { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } } peers { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } }
I20260812 03:18:16.186905 26258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0258d5c267ad4bf98506b8a768ccf9e5, State: Initialized, Role: FOLLOWER
I20260812 03:18:16.187301 26258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c5da44e4769412ba16d58d9de1f6777" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } } peers { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } } peers { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } }
I20260812 03:18:16.187680 26388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79: No bootstrap required, opened a new log
I20260812 03:18:16.189004 26258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:18:16.189056 26216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4c5da44e4769412ba16d58d9de1f6777" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0258d5c267ad4bf98506b8a768ccf9e5"
I20260812 03:18:16.189301 26216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:18:16.190754 26216 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5da44e4769412ba16d58d9de1f6777 in term 1.
I20260812 03:18:16.191227 26042 leader_election.cc:302] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0258d5c267ad4bf98506b8a768ccf9e5, 4c5da44e4769412ba16d58d9de1f6777; no voters:
I20260812 03:18:16.191540 26423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:18:16.191826 26423 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [term 1 LEADER]: Becoming Leader. State: Replica: 4c5da44e4769412ba16d58d9de1f6777, State: Running, Role: LEADER
I20260812 03:18:16.192219 26423 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c5da44e4769412ba16d58d9de1f6777" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } } peers { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } } peers { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } }
I20260812 03:18:16.194761 26388 raft_consensus.cc:348] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c5da44e4769412ba16d58d9de1f6777" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } } peers { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } } peers { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } }
I20260812 03:18:16.195209 26388 raft_consensus.cc:729] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cee10ef2e11c4f1f8dd4f873d37e6c79, State: Initialized, Role: FOLLOWER
W20260812 03:18:16.195761 26477 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:18:16.195950 26477 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:18:16.196075 26388 consensus_queue.cc:260] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c5da44e4769412ba16d58d9de1f6777" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } } peers { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } } peers { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } }
I20260812 03:18:16.198423 26342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4c5da44e4769412ba16d58d9de1f6777" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79"
I20260812 03:18:16.198681 26342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:18:16.200150 26342 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5da44e4769412ba16d58d9de1f6777 in term 1.
I20260812 03:18:16.200887 26388 sys_catalog.cc:564] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:18:16.200431 26150 master_runner.cc:186] Error getting master registration for 127.16.217.124:38901: OK, instance_id { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" instance_seqno: 1786504696122591 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:18:16.203876 26150 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:18:16.194170 26423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c5da44e4769412ba16d58d9de1f6777. Latest consensus state: current_term: 1 leader_uuid: "4c5da44e4769412ba16d58d9de1f6777" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c5da44e4769412ba16d58d9de1f6777" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } } peers { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } } peers { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } } }
I20260812 03:18:16.204058 26423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:16.206441 26481 sys_catalog.cc:455] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c5da44e4769412ba16d58d9de1f6777" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } } peers { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } } peers { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } } }
I20260812 03:18:16.206604 26481 sys_catalog.cc:458] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:18:16.210556 26450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c5da44e4769412ba16d58d9de1f6777" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } } peers { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } } peers { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } } }
I20260812 03:18:16.210700 26450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 03:18:16.211522 26500 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:18:16.211602 26500 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:18:16.213347 26145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.186556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55946 (local address 127.16.217.126:46797)
0812 03:18:16.186683 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.186687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.191929 (+ 5242us) server_negotiation.cc:407] Connection header received
0812 03:18:16.192142 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.192146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.192201 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.192257 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.193300 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.194040 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.197250 (+ 3210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.197618 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.197704 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:16.204808 (+ 7104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:16.204812 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:16.204816 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:16.204845 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:16.208734 (+ 3889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.209632 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.209638 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.209640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.209738 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.210070 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.210074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.210075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.210372 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:16.210497 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:16.213065 (+ 2568us) server_negotiation.cc:299] Negotiation successful
0812 03:18:16.213187 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:18:16.217175 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.188614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42614 (local address 127.16.217.124:38901)
0812 03:18:16.188732 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.188736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.198471 (+ 9735us) server_negotiation.cc:407] Connection header received
0812 03:18:16.198682 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.198686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.198737 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.198789 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.202331 (+ 3542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.203069 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.211311 (+ 8242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.211660 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.211736 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:16.213502 (+ 1766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:16.213506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:16.213509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:16.213535 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:16.215216 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.215752 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.215756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.215758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.215844 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.216162 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.216166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.216167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.216389 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:16.216518 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:16.216892 (+ 374us) server_negotiation.cc:299] Negotiation successful
0812 03:18:16.217015 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:18:16.218453 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:16.187769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60932 (local address 127.16.217.125:41727)
0812 03:18:16.187882 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.187888 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.195364 (+ 7476us) server_negotiation.cc:407] Connection header received
0812 03:18:16.195706 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.195747 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.196014 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.196090 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.200835 (+ 4745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.201556 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.205221 (+ 3665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:16.205610 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:16.205692 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:16.214789 (+ 9097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:18:16.214793 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:18:16.214796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:18:16.214825 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:18:16.216493 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.216976 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.216980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.216982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.217065 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:18:16.217350 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:18:16.217353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:18:16.217355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:18:16.217554 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:18:16.217684 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:16.218168 (+ 484us) server_negotiation.cc:299] Negotiation successful
0812 03:18:16.218304 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:18:16.215811 26501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:18:16.220753 26501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:18:16.226756 26342 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79 [term 1 FOLLOWER]: Refusing update from remote peer 4c5da44e4769412ba16d58d9de1f6777: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:18:16.227476 26216 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5 [term 1 FOLLOWER]: Refusing update from remote peer 4c5da44e4769412ba16d58d9de1f6777: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:18:16.227442 26423 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:18:16.228052 26423 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:18:16.231390 26450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c5da44e4769412ba16d58d9de1f6777. Latest consensus state: current_term: 1 leader_uuid: "4c5da44e4769412ba16d58d9de1f6777" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c5da44e4769412ba16d58d9de1f6777" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } } peers { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } } peers { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } } }
I20260812 03:18:16.231506 26450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:18:16.232235 26481 sys_catalog.cc:455] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c5da44e4769412ba16d58d9de1f6777. Latest consensus state: current_term: 1 leader_uuid: "4c5da44e4769412ba16d58d9de1f6777" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c5da44e4769412ba16d58d9de1f6777" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } } peers { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } } peers { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } } }
I20260812 03:18:16.232371 26481 sys_catalog.cc:458] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:18:16.237282 26478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c5da44e4769412ba16d58d9de1f6777" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c5da44e4769412ba16d58d9de1f6777" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } } peers { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } } peers { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } } }
I20260812 03:18:16.237375 26481 sys_catalog.cc:455] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c5da44e4769412ba16d58d9de1f6777" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c5da44e4769412ba16d58d9de1f6777" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } } peers { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } } peers { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } } }
I20260812 03:18:16.237444 26478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:16.237469 26481 sys_catalog.cc:458] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:18:16.237864 26504 mvcc.cc:205] Tried to move back new op lower bound from 7317523235735678976 to 7317523235607425024. Current Snapshot: MvccSnapshot[applied={T|T < 7317523235735678976}]
I20260812 03:18:16.238265 26423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c5da44e4769412ba16d58d9de1f6777" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c5da44e4769412ba16d58d9de1f6777" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } } peers { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } } peers { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } } }
I20260812 03:18:16.238377 26423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777 [sys.catalog]: This master's current role is: LEADER
I20260812 03:18:16.239892 26506 mvcc.cc:205] Tried to move back new op lower bound from 7317523235735678976 to 7317523235607425024. Current Snapshot: MvccSnapshot[applied={T|T < 7317523235735678976}]
I20260812 03:18:16.241744 26450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c5da44e4769412ba16d58d9de1f6777" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c5da44e4769412ba16d58d9de1f6777" member_type: VOTER last_known_addr { host: "127.16.217.126" port: 46797 } } peers { permanent_uuid: "0258d5c267ad4bf98506b8a768ccf9e5" member_type: VOTER last_known_addr { host: "127.16.217.125" port: 41727 } } peers { permanent_uuid: "cee10ef2e11c4f1f8dd4f873d37e6c79" member_type: VOTER last_known_addr { host: "127.16.217.124" port: 38901 } } }
I20260812 03:18:16.241863 26450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:18:16.248234 26501 catalog_manager.cc:1347] Generated new cluster ID: df7f7862d75f487ba8e8fc7526bc2e28
I20260812 03:18:16.248309 26501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:18:16.241807 26508 mvcc.cc:205] Tried to move back new op lower bound from 7317523235735678976 to 7317523235607425024. Current Snapshot: MvccSnapshot[applied={T|T < 7317523235735678976}]
I20260812 03:18:16.272369 26501 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:18:16.273685 26501 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:18:16.293453 26501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c5da44e4769412ba16d58d9de1f6777: Generated new TSK 0
I20260812 03:18:16.294422 26501 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:18:16.331224 26145 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:16.327947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55954 (local address 127.16.217.126:46797)
0812 03:18:16.328079 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.328084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.330049 (+ 1965us) server_negotiation.cc:407] Connection header received
0812 03:18:16.330298 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.330302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.330360 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.330416 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.331081 (+ 665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55954: BlockingRecv error: recv got EOF from 127.0.0.1:55954 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 03:18:16.332662 26269 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:16.328802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60946 (local address 127.16.217.125:41727)
0812 03:18:16.328915 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.328919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.331732 (+ 2813us) server_negotiation.cc:407] Connection header received
0812 03:18:16.331906 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.331910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.331960 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.332010 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.332522 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60946: BlockingRecv error: recv got EOF from 127.0.0.1:60946 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 03:18:16.334134 26406 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:16.329444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42624 (local address 127.16.217.124:38901)
0812 03:18:16.329565 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.329569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.333123 (+ 3554us) server_negotiation.cc:407] Connection header received
0812 03:18:16.333402 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.333406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.333458 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.333511 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.334016 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42624: BlockingRecv error: recv got EOF from 127.0.0.1:42624 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260812 03:18:16.338716 26269 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:16.337135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60948 (local address 127.16.217.125:41727)
0812 03:18:16.337250 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.337254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.337736 (+ 482us) server_negotiation.cc:407] Connection header received
0812 03:18:16.337958 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.337961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.338038 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.338092 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.338606 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60948: BlockingRecv error: recv got EOF from 127.0.0.1:60948 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 03:18:16.344575 26269 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:16.342866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60960 (local address 127.16.217.125:41727)
0812 03:18:16.342978 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.342983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.343559 (+ 576us) server_negotiation.cc:407] Connection header received
0812 03:18:16.343782 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.343785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.343841 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.343905 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.344455 (+ 550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60960: BlockingRecv error: recv got EOF from 127.0.0.1:60960 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 03:18:16.350615 26269 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:16.348951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60974 (local address 127.16.217.125:41727)
0812 03:18:16.349056 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.349060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.349629 (+ 569us) server_negotiation.cc:407] Connection header received
0812 03:18:16.349836 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.349839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.349898 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.349950 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.350494 (+ 544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60974: BlockingRecv error: recv got EOF from 127.0.0.1:60974 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 03:18:16.358832 26269 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:16.357011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60980 (local address 127.16.217.125:41727)
0812 03:18:16.357166 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.357171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.357753 (+ 582us) server_negotiation.cc:407] Connection header received
0812 03:18:16.357978 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.357981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.358038 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.358099 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.358717 (+ 618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60980: BlockingRecv error: recv got EOF from 127.0.0.1:60980 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260812 03:18:16.372959 26269 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:16.370693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60982 (local address 127.16.217.125:41727)
0812 03:18:16.370826 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.370831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.371756 (+ 925us) server_negotiation.cc:407] Connection header received
0812 03:18:16.371995 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.372000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.372058 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.372120 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.372836 (+ 716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60982: BlockingRecv error: recv got EOF from 127.0.0.1:60982 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 03:18:16.393849 26269 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:16.392118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60992 (local address 127.16.217.125:41727)
0812 03:18:16.392244 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.392249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.392843 (+ 594us) server_negotiation.cc:407] Connection header received
0812 03:18:16.393047 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.393051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.393108 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.393161 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.393724 (+ 563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60992: BlockingRecv error: recv got EOF from 127.0.0.1:60992 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 03:18:16.430183 26269 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:16.428681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60998 (local address 127.16.217.125:41727)
0812 03:18:16.428824 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.428830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.429098 (+ 268us) server_negotiation.cc:407] Connection header received
0812 03:18:16.429356 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.429365 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.429439 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.429571 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.430042 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60998: BlockingRecv error: recv got EOF from 127.0.0.1:60998 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 03:18:16.498063 26269 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:16.496745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32768 (local address 127.16.217.125:41727)
0812 03:18:16.496941 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.496947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.497123 (+ 176us) server_negotiation.cc:407] Connection header received
0812 03:18:16.497317 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.497321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.497374 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.497465 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.497950 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32768: BlockingRecv error: recv got EOF from 127.0.0.1:32768 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 03:18:16.629518 26269 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:16.628318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32782 (local address 127.16.217.125:41727)
0812 03:18:16.628503 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.628508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.628623 (+ 115us) server_negotiation.cc:407] Connection header received
0812 03:18:16.628802 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.628805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.628862 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.628952 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.629410 (+ 458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32782: BlockingRecv error: recv got EOF from 127.0.0.1:32782 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 03:18:16.889900 26269 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:18:16.888563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32790 (local address 127.16.217.125:41727)
0812 03:18:16.888709 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:16.888713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:16.888728 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:18:16.888789 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:16.888792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:16.888842 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:16.888945 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:18:16.889695 (+ 750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32790: BlockingRecv error: recv got EOF from 127.0.0.1:32790 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:18:17.200002 26477 catalog_manager.cc:1259] Loaded cluster ID: df7f7862d75f487ba8e8fc7526bc2e28
I20260812 03:18:17.200094 26477 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5: loading cluster ID for follower catalog manager: success
I20260812 03:18:17.201866 26477 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5: acquiring CA information for follower catalog manager: success
I20260812 03:18:17.203125 26477 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0258d5c267ad4bf98506b8a768ccf9e5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:18:17.212829 26500 catalog_manager.cc:1259] Loaded cluster ID: df7f7862d75f487ba8e8fc7526bc2e28
I20260812 03:18:17.212901 26500 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79: loading cluster ID for follower catalog manager: success
I20260812 03:18:17.214622 26500 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79: acquiring CA information for follower catalog manager: success
I20260812 03:18:17.215801 26500 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cee10ef2e11c4f1f8dd4f873d37e6c79: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:18:17.408036 26634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:17.404163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32800 (local address 127.16.217.125:41727)
0812 03:18:17.404413 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:17.404417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:17.404434 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:18:17.404519 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:17.404522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:17.404675 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:17.404805 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:18:17.405072 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:17.405613 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:17.406569 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:17.406813 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:17.406901 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:17.407225 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:18:17.407671 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:18:17.407780 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:17.407857 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 03:18:17.407907 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":97,"threads_started":1}
I20260812 03:18:17.414997 26644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:17.411255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42630 (local address 127.16.217.124:38901)
0812 03:18:17.411542 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:17.411546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:17.411560 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:18:17.411646 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:17.411649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:17.411774 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:17.411886 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:18:17.412136 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:17.412670 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:17.413499 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:17.413724 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:17.413806 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:17.414083 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:18:17.414583 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:18:17.414676 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:17.414748 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 03:18:17.414798 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":98,"threads_started":1}
I20260812 03:18:17.422945 26644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:17.419221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42638 (local address 127.16.217.124:38901)
0812 03:18:17.419595 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:17.419600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:17.419621 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:18:17.419671 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:17.419674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:17.419721 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:17.419818 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:18:17.420251 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:17.420869 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:17.421741 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:17.421958 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:17.422067 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:17.422401 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:18:17.422468 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:18:17.422552 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:17.422719 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 03:18:17.422784 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260812 03:18:17.423195 26634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:17.418670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32812 (local address 127.16.217.125:41727)
0812 03:18:17.418857 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:17.418863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:17.419095 (+ 232us) server_negotiation.cc:407] Connection header received
0812 03:18:17.419312 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:17.419316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:17.419378 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:17.419435 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:18:17.419773 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:17.420528 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:17.421761 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:17.422121 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:17.422200 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:17.422661 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:18:17.422738 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:18:17.422809 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:17.422995 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 03:18:17.423048 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 03:18:17.424081 26658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:18:17.418598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55956 (local address 127.16.217.126:46797)
0812 03:18:17.418848 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 03:18:17.418854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:18:17.418958 (+ 104us) server_negotiation.cc:407] Connection header received
0812 03:18:17.419596 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:18:17.419600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:18:17.419780 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:18:17.419943 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:18:17.420198 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:17.420858 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:17.422115 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:18:17.422492 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:18:17.422605 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:18:17.423083 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:18:17.423550 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:18:17.423659 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:18:17.423784 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 03:18:17.423871 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":70,"threads_started":1}
I20260812 03:18:17.427624 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 26012
I20260812 03:18:17.460628 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 26150
I20260812 03:18:17.477233 17253 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm8e4iX/build/debug/bin/kudu with pid 26271
2026-08-12T03:18:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3452 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6883 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (314504 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST